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Abstract — This  paper  explores  the  idea  of  knowledge-based 
security  policies,  which  are  used  to  decide  whether  to  answer 
queries  over  secret  data  based  on  an  estimation  of  the  querier’s 
(possibly  increased)  knowledge  given  the  results.  Limiting 
knowledge  is  the  goal  of  existing  information  release  poli¬ 
cies  that  employ  mechanisms  such  as  noising,  anonymization, 
and  redaction.  Knowledge-based  policies  are  more  general: 
they  increase  flexibility  by  not  fixing  the  means  to  restrict 
information  flow.  We  enforce  a  knowledge-based  policy  by 
explicitly  tracking  a  model  of  a  querier’s  belief  about  secret 
data,  represented  as  a  probability  distribution,  and  denying  any 
query  that  could  increase  knowledge  above  a  given  threshold. 
We  implement  query  analysis  and  belief  tracking  via  abstract 
interpretation  using  a  novel  probabilistic  polyhedral  domain, 
whose  design  permits  trading  off  precision  with  performance 
while  ensuring  estimates  of  a  querier’s  knowledge  are  sound. 
Experiments  with  our  implementation  show  that  several  useful 
queries  can  be  handled  efficiently,  and  performance  scales 
far  better  than  would  more  standard  implementations  of 
probabilistic  computation  based  on  sampling. 

I.  Introduction 

Facebook,  Twitter,  Flickr,  and  other  successful  on-line  ser¬ 
vices  enable  users  to  easily  foster  and  maintain  relationships 
by  sharing  information  with  friends  and  fans.  These  services 
store  users’  personal  information  and  use  it  to  customize  the 
user  experience  and  to  generate  revenue.  For  example.  Face- 
book  third-party  applications  are  granted  access  to  a  user’s 
“basic”  data  (which  includes  name,  profile  picture,  gender, 
networks,  user  ID,  and  list  of  friends  [1])  to  implement 
services  like  birthday  announcements  and  horoscopes,  while 
Facebook  selects  ads  based  on  age,  gender,  and  even  sexual 
preference  [2],  Unfortunately,  once  personal  information  is 
collected,  users  have  limited  control  over  how  it  is  used. 
For  example,  Facebook’s  EULA  grants  Facebook  a  non¬ 
exclusive  license  to  any  content  a  user  posts  [3],  MySpace, 
another  social  network  site,  has  recently  begun  to  sell  its 
users’  data  [4], 

Some  researchers  have  proposed  that,  to  keep  tighter 
control  over  their  data,  users  could  use  a  storage  server 
(e.g.,  running  on  their  home  network)  that  handles  personal 
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data  requests,  and  only  responds  when  a  request  is  deemed 
safe  [5],  [6].  The  question  is:  which  requests  are  safe?  While 
deferring  to  user-defined  access  control  policies  seems  an 
obvious  approach,  such  policies  are  unnecessarily  restrictive 
when  the  goal  is  to  maximize  the  customized  personal 
experience.  To  see  why,  consider  two  example  applications: 
a  horoscope  or  “happy  birthday”  application  that  operates 
on  birth  month  and  day,  and  a  music  recommendation 
algorithm  that  considers  birth  year  (age).  Access  control  at 
the  granularity  of  the  entire  birth  date  could  preclude  both 
of  these  applications,  while  choosing  only  to  release  birth 
year  or  birth  day  precludes  access  to  one  application  or  the 
other.  But  in  fact  the  user  may  not  care  much  about  these 
particular  bits  of  information,  but  rather  about  what  can  be 
deduced  from  them.  For  example,  it  has  been  reported  that 
zip  code,  birth  date,  and  gender  are  sufficient  information  to 
uniquely  identify  63%  of  Americans  in  the  2000  U.S.  census 
[7],  So  the  user  may  be  perfectly  happy  to  reveal  any  one  of 
these  bits  of  information  in  its  entirety  as  long  as  a  querier 
gains  no  better  than  a  1/n  chance  to  guess  the  entire  group, 
for  some  parameter  n. 

This  paper  explores  the  design  and  implementation  for 
enforcing  what  we  call  knowledge-based  security  policies.  In 
our  model,  a  user  U’s  agent  responds  to  queries  involving  se¬ 
cret  data.  For  each  querying  principal  Q ,  the  agent  maintains 
a  probability  distribution  over  U’s  secret  data,  representing 
Q’s  belief  of  the  data’s  likely  values.  For  example,  to 
mediate  queries  from  a  social  networking  site  X,  user  U’s 
agent  may  model  X’s  otherwise  uninformed  knowledge  of 
U’s  birthday  according  to  a  likely  demographic:  the  birth 
month  and  day  are  uniformly  distributed,  while  the  birth 
year  is  most  likely  between  1956  and  1992  [8].  Each  querier 
Q  is  also  assigned  a  knowledge-based  policy,  expressed 
as  a  set  of  thresholds,  each  applying  to  a  different  group 
of  (potentially  overlapping)  data.  For  example,  U’s  policy 
for  X  might  be  a  threshold  of  1/100  for  the  entire  tuple 
( birthdate ,  zipcode,  gender),  and  1/5  for  just  birth  date.  U’s 
agent  refuses  any  queries  that  it  determines  could  increase 
Q’s  ability  to  guess  a  secret  above  the  assigned  threshold.  If 
deemed  safe,  U’s  agent  returns  the  query’s  (exact)  result  and 
updates  Q’s  modeled  belief  appropriately.  (We  touch  upon 
the  risk  of  colluding  queriers  shortly.) 

To  implement  our  model,  we  need  (1)  an  algorithm  to 
check  whether  answering  a  query  could  violate  a  knowledge- 
based  policy,  (2)  a  method  for  revising  a  querier’s  belief 
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according  to  the  answer  that  is  given,  and  (3)  means  to 
implement  (1)  and  (2)  efficiently.  We  build  on  the  work  of 
Clarkson  et  al.  [9]  (reviewed  in  Section  III),  which  works  out 
the  theoretical  basis  for  (2).  The  main  contributions  of  this 
paper,  therefore,  in  addition  to  the  idea  of  knowledge-based 
policies,  are  our  solutions  to  problems  (1)  and  (3). 

Given  a  means  to  revise  querier  beliefs  based  on  prior 
answers,  it  seems  obvious  how  to  check  that  a  query  does 
not  reveal  too  much:  U  runs  the  query,  tentatively  revises 
Q's  belief  based  on  the  result,  and  then  responds  with  the 
answer  only  if  Q’s  revised  belief  about  the  secrets  does  not 
exceed  the  prescribed  thresholds.  Unfortunately,  with  this 
approach  the  decision  to  deny  depends  on  the  actual  secret, 
so  a  rejection  could  leak  information.  We  give  an  example 
in  the  next  section  that  shows  how  the  entire  secret  could 
be  revealed.  Therefore,  we  propose  that  a  query  should  be 
rejected  if  there  exists  any  possible  secret  value  that  could 
induce  an  output  whereby  the  revised  belief  would  exceed 
the  threshold.  This  idea  is  described  in  detail  in  Section  IV. 

To  implement  belief  tracking  and  revision,  our  first 
thought  was  to  use  languages  for  probabilistic  computation 
and  conditioning,  which  provide  the  foundational  elements 
of  the  approach.  Languages  we  know  of — IBAL  [10],  Prob¬ 
abilistic  Scheme  [11],  and  several  other  systems  [12],  [13], 
[14] — are  implemented  using  sampling.  Unfortunately,  we 
found  these  implementations  to  be  inadequate  because  they 
either  underestimate  the  querier’s  knowledge  when  sampling 
too  little,  or  run  too  slowly  when  the  state  space  is  large. 

Instead  of  using  sampling,  we  have  developed  an  imple¬ 
mentation  based  on  abstract  interpretation.  In  Section  V  we 
develop  a  novel  abstract  domain  of  probabilistic  polyhedra, 
which  extends  the  standard  convex  polyhedron  abstract 
domain  [15]  with  measures  of  probability.  We  represent 
beliefs  as  a  set  of  probabilistic  (as  developed  in  Section  VI). 
While  some  prior  work  has  explored  probabilistic  abstract 
interpretation  [16],  this  work  does  not  support  belief  revi¬ 
sion,  which  is  required  to  track  how  observation  of  out¬ 
puts  affects  a  querier’s  belief.  Support  for  revision  requires 
that  we  maintain  both  under-  and  over-approximations  of 
the  querier’s  belief,  whereas  [16]  deals  only  with  over¬ 
approximation.  We  have  developed  an  implementation  of 
our  approach  based  on  Parma  [17]  and  LattE  [18],  which 
we  present  in  Section  VII  along  with  some  experimental 
measurements  of  its  performance.  We  find  that  while  the 
performance  of  Probabilistic  Scheme  degrades  significantly 
as  the  input  space  grows,  our  implementation  scales  much 
better,  and  can  be  orders  of  magnitude  faster. 

Knowledge-based  policies  aim  to  ensure  that  an  attacker’s 
knowledge  of  a  secret  does  not  increase  much  when  learning 
the  result  of  a  query.  Much  prior  work  aims  to  enforce 
similar  properties  by  tracking  information  leakage  quantita¬ 
tively  [19],  [20],  [21],  [22],  [23].  Our  approach  is  more  pre¬ 
cise  (but  also  more  resource-intensive)  because  it  maintains 
an  on-line  model  of  adversary  knowledge.  An  alternative  to 


knowledge-based  privacy  is  differential  privacy  [24]  (DP), 
which  requires  that  a  query  over  a  database  of  individu¬ 
als’  records  produces  roughly  the  same  answer  whether  a 
particular  individual’s  data  is  in  the  database  or  not — the 
possible  knowledge  of  the  querier,  and  the  impact  of  the 
query’s  result  on  it,  need  not  be  directly  considered.  As  such, 
DP  avoids  the  danger  of  mismodeling  a  querier’s  knowledge 
and  as  a  result  inappropriately  releasing  information.  DP  also 
ensures  a  high  degree  of  compositionality,  which  provides 
some  assurance  against  collusion.  However,  DP  applies  once 
an  individual  has  released  his  personal  data  to  a  trusted 
third  party’s  database,  a  release  we  are  motivated  to  avoid. 
Moreover,  applying  DP  to  queries  over  an  individual’s  data, 
rather  than  a  population,  introduces  so  much  noise  that  the 
results  are  often  useless.  We  discuss  these  issues  along  with 
other  related  work  in  Section  VIII. 

The  next  section  presents  a  technical  overview  of  the 
rest  of  the  paper,  whose  main  results  are  contained  in 
Sections  III— VII,  with  further  discussion  and  ideas  for  future 
work  in  Sections  VIII  and  IX. 

II.  Overview 

Knowledge-based  policies  and  beliefs.  User  Bob  would 
like  to  enforce  a  knowledge-based  policy  on  his  data  so  that 
advertisers  do  not  learn  too  much  about  him.  Suppose  Bob 
considers  his  birthday  of  September  27,  1980  to  be  relatively 
private;  variable  bday  stores  the  calendar  day  (a  number 
between  0  and  364,  which  for  Bob  would  be  270)  and  byear 
stores  the  birth  year  (which  would  be  1980).  To  bday  he 
assigns  a  knowledge  threshold  td  =  0.2  stating  that  he  does 
not  want  an  advertiser  to  have  better  than  a  20%  likelihood 
of  guessing  his  birth  day.  To  the  pair  (bday,  by  ear)  he 
assigns  a  threshold  tdy  =  0.05,  meaning  he  does  not  want 
an  advertiser  to  be  able  to  guess  the  combination  of  birth 
day  and  year  together  with  better  than  a  5%  likelihood. 

Bob  runs  an  agent  program  to  answer  queries  about 
his  data  on  his  behalf.  This  agent  models  an  estimated 
belief  of  queriers  as  a  probability  distribution  S,  which 
is  conceptually  a  map  from  secret  states  to  positive  real 
numbers  representing  probabilities  (in  range  [0, 1]).  Bob’s 
secret  state  is  the  pair  ( bday  =  270,  byear  =  1980) .  The  agent 
represents  a  distribution  as  a  set  of  probabilistic  polyhedra. 
For  now,  we  can  think  of  a  probabilistic  polyhedron  as  a 
standard  convex  polyhedron  C  with  a  probability  mass  to, 
where  the  probability  of  each  integer  point  contained  in  C 
is  m/ff(C),  where  ff(C)  is  the  number  of  integer  points 
contained  in  the  polyhedron  C.  Shortly  we  present  a  more 
involved  representation. 

Initially,  the  agent  might  model  an  advertiser  X’s  belief 
using  the  following  rectangular  polyhedron  C,  where  each 
point  contained  in  it  is  considered  equally  likely  (to  =  1): 

C  =  0  <  bday  <  365,  1956  <  byear  <  1993 
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Enforcing  knowledge-based  policies  safely.  Suppose  X 
wants  to  identify  users  whose  birthday  falls  within  the  next 
week,  to  promote  a  special  offer.  X  sends  Bob’s  agent  the 
following  program. 

Example  1. 

today  :=  260; 

if  bday  >  today  A  bday  <  ( today  +  7)  then 
output  :=  True; 

This  program  refers  to  Bob’s  secret  variable  bday ,  and  also 
uses  non-secret  variables  today,  which  represents  the  current 
day  and  is  here  set  to  be  260,  and  output,  which  is  set  to 
True  if  the  user’s  birthday  is  within  the  next  seven  days  (we 
assume  output  is  initially  False). 

The  agent  must  decide  whether  returning  the  result  of  run¬ 
ning  this  program  will  potentially  increase  X’s  knowledge 
about  Bob’s  data  above  the  prescribed  threshold.  We  explain 
how  it  makes  this  determination  shortly,  but  for  the  present 
we  can  see  that  answering  the  query  is  safe:  the  returned 
output  variable  will  be  False  which  essentially  teaches  the 
querier  that  Bob’s  birthday  is  not  within  the  next  week, 
which  still  leaves  many  possibilities.  As  such,  the  agent 
revises  his  model  of  the  querier’s  belief  to  be  the  following 
pair  of  rectangular  polyhedra  Ci,  C-i ,  where  again  all  points 
in  each  are  equally  likely  (m,i  ~  0.726,  m2  ~  0.274): 

C\  =  0  <  bday  <  260,  1956  <  byear  <  1993 
C2  =  267  <  bday  <  365,  1956  <  byear  <  1993 

Ignoring  byear,  there  are  358  possible  values  for  bday  and 
each  is  equally  likely.  Thus  the  probability  of  any  one  is 
1/358  ~  0.0028  <td  =  0.2. 

Suppose  the  next  day  the  same  advertiser  sends  the  same 
program  to  Bob’s  user  agent,  but  with  today  set  to  261. 
Should  the  agent  run  the  program?  At  first  glance,  doing  so 
seems  OK.  The  program  will  return  False,  and  the  revised 
belief  will  be  the  same  as  above  but  with  constraint  bday  > 
267  changed  to  bday  >  268,  meaning  there  is  still  only  a 
1/357  =  0.0028  chance  to  guess  bday. 

But  suppose  Bob’s  birth  day  was  actually  267,  rather  than 
270.  The  first  query  would  have  produced  the  same  revised 
belief  as  before,  but  since  the  second  query  would  return 
True  (since  bday  =  267  <  (261+7)),  the  querier  can  deduce 
Bob’s  birth  day  exactly:  bday  >  267  (from  the  first  query) 
and  bday  <  268  (from  the  second  query)  together  imply 
that  bday  =  267!  But  the  user  agent  is  now  stuck:  it  cannot 
simply  refuse  to  answer  the  query,  because  the  querier  knows 
that  with  td  =  0.2  (or  indeed,  any  reasonable  threshold)  the 
only  good  reason  to  refuse  is  when  bday  =  267.  As  such, 
refusal  essentially  tells  the  querier  the  answer. 

The  lesson  is  that  the  decision  to  refuse  a  query  must  not 
be  based  on  the  effect  of  running  the  query  on  the  actual 
secret,  because  then  a  refusal  could  leak  information.  In 
Section  IV  we  propose  that  an  agent  should  reject  a  program 
if  there  exists  any  possible  secret  that  could  cause  a  program 
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(a)  output  =  False  (b)  output  =  True 

Figure  1.  Example  2:  most  precise  revised  beliefs 


answer  to  increase  querier  knowledge  above  the  threshold. 
As  such  we  would  reject  the  second  query  regardless  of 
whether  bday  =  270  or  bday  =  267. 

Full  probabilistic  polyhedra.  Now  suppose,  having  run 
the  first  query  and  rejected  the  second,  the  user  agent 
receives  the  following  program  from  X. 

Example  2. 

age  :=  2011  —  byear\ 
if  age  =  20  V  ...  V  age  =  60  then 
output  :=  True; 
pif  0.1  then  output  :=  True; 

This  program  attempts  to  discover  whether  this  year  is  a 
“special”  year  for  the  given  user,  who  thus  deserves  a  special 
offer.  The  program  returns  True  if  either  the  user’s  age  is 
(or  will  be)  an  exact  decade,  or  if  the  user  wins  the  luck 
of  the  draw  (one  chance  in  ten),  as  implemented  by  the 
probabilistic  if  statement. 

Running  this  program  reveals  nothing  about  bday, 
but  does  reveal  something  about  byear.  In  particular,  if 
output  =  False  then  the  querier  knows  that  byear  qL 
{1991,1981,1971,1961},  but  all  other  years  are  equally 
likely.  We  could  represent  this  new  knowledge,  combined 
with  the  knowledge  gained  from  the  first  query,  as  shown 
in  Figure  1(a),  where  each  shaded  box  is  a  polyhedron  con¬ 
taining  equally  likely  points.  On  the  other  hand,  if  output  = 
True  then  either  byear  €  {1991,1981,1971,1961}  or  the 
user  got  lucky.  We  represent  the  querier’s  knowledge  in 
this  case  as  in  Figure  1(b).  Darker  shading  indicates  higher 
probability;  thus,  all  years  are  still  possible,  though  some 
are  much  more  likely  than  others.  With  the  given  threshold 
of  tdy  =  0.05,  the  agent  will  permit  the  query;  when 
output  =  False,  the  likelihood  of  any  point  in  the  shaded  re¬ 
gion  is  1/11814;  when  output  =  True,  the  points  in  the  dark 
bands  are  the  most  likely,  with  probability  5/13067.  Since 
both  outcomes  are  possible  with  Bob’s  byear  =  1980,  the 
revised  belief  will  depend  on  the  result  of  the  probabilistic 
if  statement. 

This  example  illustrates  a  potential  problem  with  the 
simple  representation  of  probabilistic  polyhedra  mentioned 
earlier:  when  output  =  False  we  will  jump  from  using  two 
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probabilistic  polyhedra  to  ten,  and  when  output  =  True  we 
jump  to  using  eighteen.  Allowing  the  number  of  polyhedra 
to  grow  without  bound  will  result  in  performance  problems. 
To  address  this  concern,  we  need  a  way  to  abstract  our 
belief  representation  to  be  more  concise.  Section  V  shows 
how  to  represent  a  probabilistic  polyhedron  P  as  a  seven¬ 
tuple,  (C,  smin,  smax,  pmin,  pmax,  mmin,  mmax)  where  smin 
and  smax  are  lower  and  upper  bounds  on  the  number  of 
points  with  non-zero  probability  in  the  polyhedron  C  (called 
the  support  points  of  (7);  the  quantities  pmin  and  pmax 
are  lower  and  upper  bounds  on  the  probability  mass  per 
support  point;  and  mmin  and  mmax  give  bounds  on  the  total 
probability  mass.  Thus,  polyhedra  modeled  using  the  simpler 
representation  (C,  m)  given  earlier  are  equivalent  to  ones  in 
the  more  involved  representation  with  mmax  =  mmin  =  m, 
pmax  =  pmin  =  m/#(C),  and  SmaX  =  Smin  =  #(C). 

With  this  representation,  we  could  choose  to  collapse 
the  sets  of  polyhedron  given  in  Figure  1.  For  example,  we 
could  represent  Figure  1(a)  with  two  probabilistic  polyhe¬ 
dra  Pi  and  P2  containing  polyhedra  C\  and  C2  defined 
above,  respectively,  essentially  drawing  a  box  around  the 
two  groupings  of  smaller  boxes  in  the  figure.  The  other 
parameters  for  Pi  would  be  as  follows; 

pmin  =  pmax  =  9/135Q50 
gmin  =  smax  =  gggg 

mjnin  =  m”lax  =  7722/13505 

Notice  that  sfm  =  s^ax  =  8580  <  #(Ci)  =  9620, 
illustrating  that  the  “bounding  box”  of  the  polyhedron  covers 
more  area  than  is  strictly  necessary.  In  this  representation  the 
probabilities  may  not  be  normalized,  which  improves  both 
performance  and  precision.  For  this  example,  P2  happens 
to  have  m™ln  =  m™ax  =  14553/67525  so  we  can  see 
mjnax  +  m“ax  =  (53163/67525)  ^  1. 

If  we  consider  the  representation  of  Figure  1(b)  in  a 
similar  manner,  using  the  same  two  polyhedra  Ci  and  C2 , 
the  other  parameters  for  Ci  are  as  follows; 

pfn  =  1/135050  p^ax  =  10/135050 
sfn  =  9620  s5“iax  =  9620 
m5nin  =  26/185  m5nax  =  26/185 

In  this  case  sjnln  =  s“ax  =  ^(C 1),  meaning  that  all  covered 
points  are  possible,  but  pjnm  ^  p™ax  as  some  points  are 
more  probable  than  others  (i.e.,  those  in  the  darker  band). 

The  key  property  of  probabilistic  polyhedra,  and  a  main 
technical  contribution  of  this  paper,  is  that  this  abstraction 
can  be  used  to  make  sound  security  policy  decisions.  To 
accept  a  query,  we  must  check  that,  for  all  possible  outputs, 
the  querier’s  revised,  normalized  belief  of  any  of  the  possible 
secrets  is  below  the  threshold  t.  In  checking  whether  the 
revised  beliefs  in  our  example  are  acceptable,  the  agent  will 
try  to  find  the  maximum  probability  the  querier  could  ascribe 
to  a  state,  for  each  possible  output.  In  the  case  output  = 
True,  the  most  probable  points  are  those  in  the  dark  bands. 
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which  each  have  probability  mass  10/135050  =  p™ax  (the 
dark  bands  in  P2  have  the  same  probability).  To  find  the 
maximum  normalized  probability  of  these  points,  we  divide 
by  the  minimum  possible  total  mass,  as  given  by  the  lower 
bounds  in  our  abstraction.  In  our  example,  this  results  in 
pr/fmf+m™)  =  (10/135050) / (26/185+49/925)  « 
0.0004  <td  =  0.05. 

As  just  shown,  the  bound  on  minimum  total  mass  is 
needed  in  order  to  soundly  normalize  distributions  in  our 
abstraction.  The  maintenance  of  such  lower  bounds  on 
probability  mass  is  a  key  component  of  our  abstraction  that 
is  missing  from  prior  work.  Each  of  the  components  of  a 
probabilistic  polyhedron  play  a  role  in  producing  the  lower 
bound  on  total  mass.  While  s™m, s™ax, p™ln,  and  m™ax  do 
not  play  a  role  in  making  the  final  policy  decision,  their 
existence  allows  us  to  more  accurately  update  belief  during 
the  query  evaluation  that  precedes  the  final  policy  check. 
The  choice  of  the  number  of  probabilistic  polyhedra  to  use 
impacts  both  precision  and  performance,  so  choosing  the 
right  number  is  a  challenge.  For  the  examples  given  in  this 
section,  our  implementation  can  often  answer  queries  in  a 
few  seconds;  details  are  in  Sections  V-VII. 

III.  Tracking  beliefs 

This  section  reviews  Clarkson  et  al.’s  method  of  revising  a 
querier’s  belief  of  the  possible  valuations  of  secret  variables 
based  on  the  result  of  a  query  involving  those  variables  [9]. 

A.  Core  language 

The  programming  language  we  use  for  queries  is  given  in 
Figure  2.  A  computation  is  defined  by  a  statement  S  whose 
standard  semantics  can  be  viewed  as  a  relation  between 
states;  given  an  input  state  <r,  running  the  program  will 
produce  an  output  state  o' .  States  are  maps  from  variables 
to  integers: 

o,  T  G  State  =  Yar  -►  Z 

Sometimes  we  consider  states  with  domains  restricted  to 
a  subset  of  variables  V,  in  which  case  we  write  oy  G 
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Statey  =  V  — »  Z.  We  may  also  project  states  to  a  set 
of  variables  V: 

a  I"  V  =  Xx  G  Vary.  cr(a;) 

The  language  is  essentially  standard.  We  limit  the  form 
of  expressions  to  support  our  abstract  interpretation-based 
semantics  (Section  V).  The  semantics  of  the  statement  form 
pif  q  then  S±  else  S2  is  non-deterministic:  the  result  is  that 
of  Si  with  probability  q,  and  S2  with  probability  1  —  q. 

B.  Probabilistic  semantics  for  tracking  beliefs 

To  enforce  a  knowledge-based  policy,  a  user  agent  must  be 
able  to  estimate  what  a  querier  could  learn  from  the  output 
of  his  query.  To  do  this,  the  agent  keeps  a  distribution  8  that 
represents  the  querier’s  belief  of  the  likely  valuations  of  the 
user’s  secrets.  More  precisely,  a  distribution  is  a  map  from 
states  to  positive  real  numbers,  interpreted  as  probabilities 
(in  range  [0,1]). 

8  G  Dist  =  State  — >  M+ 

We  sometimes  focus  our  attention  on  distributions  over  states 
of  a  fixed  set  of  variables  V,  in  which  case  we  write  8y  G 
Disty  to  mean  Statey  — »  R+.  Projecting  distributions  onto 
a  set  of  variables  is  as  follows:1 

<5  |  V  =  Aery  g  Statey. 

cr'\(cr'\V=crv) 

The  mass  of  a  distribution,  written  ||<5||  is  the  sum  of  the 
probabilities  ascribed  to  states,  8(a).  A  normalized  dis¬ 
tribution  is  one  such  that  ||<5||  =  1.  A  normalized  distribution 
can  be  constructed  by  scaling  a  distribution  according  to  its 
mass: 

normal (5)  =  p||  ’  6 

The  support  of  a  distribution  is  the  set  of  states  which  have 
non-zero  probability:  support(8)  =  {cr  |  8(a)  >  0}. 

The  agent  evaluates  a  query  in  light  of  the  querier’s  initial 
belief  using  a  probabilistic  semantics.  Figure  3  defines  a 
semantic  function  [■]  whereby  [S'] <5  =  8'  indicates  that, 
given  an  input  distribution  8 ,  the  semantics  of  program  S 
is  the  output  distribution  8' .  The  semantics  is  defined  in 
terms  of  operations  on  distributions,  including  assignment 
8  [v  — >  E]  (used  in  the  rule  for  v  :=  E ),  conditioning  8\B 
and  addition  (5i  +  82  (used  in  the  rule  for  if),  and  scaling 
q  ■  8  where  q  is  a  rational  (used  for  pif).  The  semantics  is 
standard  (cf.  Clarkson  et  al.  [9]).  A  brief  review  is  given  in 
Appendix  A. 

The  notation  p  can  be  read  p  is  the  sum  over  all  x  such  that 

formula  n  is  satisfied  (where  x  is  bound  in  p  and  it). 


=  8 

=  8  [x  — »  E] 

=  I&PIBj  +  I&KJhB) 

=  lSil(q-  <5)  +  [S2]((l  -  q)  ■  6) 

=  [&]([SiM) 

=  lfp  [A/  :  Dist  — >  Dist.  A<5. 
f(lS](S\B))  +  (ShB)} 

where 

5[x-*E\  =  Act.  £r  I  T[*-[£]T]=<r  S(T) 

81  +  82  =  Act.  <5i(ct)  +  82(a) 

8\B  =  Act.  if  \B\a  then  8(a)  else  0 

p  ■  8  =f  Act.  p  ■  8(a) 

Figure  3.  Probabilistic  semantics  for  the  core  language 

C.  Belief  and  security 

Clarkson  et  al.  [9]  describe  how  a  belief  about  possible 
values  of  a  secret,  expressed  as  a  probability  distribution, 
can  be  revised  according  to  an  experiment  using  the  actual 
secret.  Such  an  experiment  works  as  follows. 

The  values  of  the  set  of  secret  variables  H  are  given  by 
the  hidden  state  an-  The  attacker’s  initial  belief  as  to  the 
possible  values  of  a h  is  represented  as  a  distribution  8 u . 

A  query  is  a  program  S  that  makes  use  of  variables  H  and 
possibly  other,  non-secret  variables  from  a  set  L;  the  final 
values  of  L.  after  running  S,  are  made  visible  to  the  attacker. 
Let  a  1  be  an  arbitrary  initial  state  of  these  variables  such 
that  domain(a l)  =  L.  Then  we  take  the  following  steps: 

Step  1.  Evaluate  S  probabilistically  using  the  attacker’s 
belief  about  the  secret  to  produce  an  output  distribution  8' , 
which  amounts  to  the  attacker’s  prediction  of  the  possible 
output  states.  This  is  computed  as  8'  =  [5] <5,  where  8,  a 
distribution  over  variables  fTl+lL,  is  defined  as  8  =  Sh  xctl- 
Here,  we  make  use  of  the  distribution  product  operator  and 
point  operator.  That  is,  given  5i,  cL,  which  are  distributions 
over  states  having  disjoint  domains,  the  distribution  product 
is 

x  =  A(cti,ct2).  <AlOi)  ■  £2(02) 

where  (or,  02)  is  the  “concatenation”  of  the  two  states, 
which  is  itself  a  state  and  is  well-defined  because  the  two 
states’  domains  are  disjoint.  And,  given  a  state  ct,  the  point 
distribution  ct  is  a  distribution  in  which  only  a  is  possible: 

ct  =f  At.  if  ct  =  t  then  1  else  0 

Thus,  the  initial  distribution  8  is  the  attacker’s  belief  about 
the  secret  variables  combined  with  an  arbitrary  valuation  of 
the  public  variables. 

Step  2.  Using  the  actual  secret  an,  evaluate  S  “con¬ 
cretely”  to  produce  an  output  state  tf^,  in  three  steps.  First, 
we  have  S'  =  [S'] 8,  where  <5  =  &h  x  dp.  Second,  we  have 
ct  G  T(S)  where  T  is  a  sampling  operator  that  produces  a 
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state  a  from  the  domain  of  a  distribution  8  with  probability 
<5(cr)/||<5||.  Finally,  we  extract  the  attacker-visible  output  of 
the  sampled  state  by  projecting  away  the  high  variables: 
<7l  =  <7  \  L. 

Step  3.  Revise  the  attacker’s  initial  belief  d#  according  to 
the  observed  output  <5x,  yielding  a  new  belief  5h  =  S'\dL  \ 
H.  Here,  S'  is  conditioned  on  the  output  <5x,  which  yields 
a  new  distribution,  and  this  distribution  is  then  projected  to 
the  variables  H.  The  conditioning  operation  is  defined  as 
follows: 

8\(Jv  =  Act.  if  a  \  V  =  ay  then  8(a)  else  0 

Note  that  this  protocol  assumes  that  S  always  terminates 
and  does  not  modify  the  secret  state.  The  latter  assumption 
can  be  eliminated  by  essentially  making  a  copy  of  the  state 
before  running  the  program,  while  eliminating  the  former  de¬ 
pends  on  the  observer’s  ability  to  detect  nontermination  [9], 

IV.  Enforcing  knowledge-based  policies 

When  presented  with  a  query  over  a  user’s  data  an,  the 
user’s  agent  should  only  answer  the  query  if  doing  so  will 
not  reveal  too  much  information.  More  precisely,  given  a 
query  S,  the  agent  will  return  the  public  output  ax  resulting 
from  running  S  on  an  if  the  agent  deems  that  from  this 
output  the  querier  cannot  guess  the  secret  state  a #  beyond 
some  level  of  doubt,  identified  by  a  threshold  t.  If  this 
threshold  could  be  exceeded,  then  the  agent  declines  to  run 
S.  We  call  this  security  check  knowledge  threshold  security. 

Definition  3  (Knowledge  Threshold  Security).  Let  8'  = 
[5]  8,  where  8  is  the  model  of  the  querier’s  initial  be¬ 
lief.  Then  query  S  is  threshold  secure  iff  for  all  ox  £ 
supported’  I"  L)  and  all  a'H  £  State#  we  have 
( normal ((8' | ox)  f  H))(o'h)  <  t  for  some  threshold  t. 

This  definition  can  be  related  to  the  experiment  protocol 
defined  in  Section  III-C.  First,  S'  in  the  definition  is  the  same 
as  S'  computed  in  the  first  step  of  the  protocol.  Step  2  in  the 
protocol  produces  a  concrete  output  ox  based  on  executing 
S  on  the  actual  secret  a#,  and  Step  3  revises  the  querier’s 
belief  based  on  this  output.  Definition  3  generalizes  these 
two  steps:  instead  of  considering  a  single  concrete  output 
based  on  the  actual  secret  it  considers  all  possible  concrete 
outputs,  as  given  by  support  (S'  \  L ),  and  ensures  that  the 
revised  belief  in  each  case  for  all  possible  secret  states  must 
assign  probability  no  greater  than  t. 

This  definition  considers  a  threshold  for  the  whole  secret 
state  a#.  As  described  in  Section  II  we  can  also  enforce 
thresholds  over  portions  of  a  secret  state.  In  particular,  a 
threshold  that  applies  only  to  variables  V  C  H  requires  that 
all  a'v  £  Statey  result  in  (normal(c)'|ox  \  V))(oy)  <  t. 

The  two  “foralls”  in  the  definition  are  critical  for  ensuring 
security.  The  reason  was  shown  by  the  first  example  in 
Section  II:  If  we  used  the  flawed  approach  of  just  running 
the  experiment  protocol  and  checking  if  >  t 


then  rejection  depends  on  the  value  of  the  secret  state  and 
could  reveal  information  about  it.  The  more  general  policy 
Vox  €  support(8'  f  L).  (normal^' |  ox  f  H))(uh)  <  t, 
would  sidestep  the  problem  in  the  example,  but  this  policy 
could  still  reveal  information  because  it,  too,  depends  on 
the  actual  secret  a#.  (An  example  illustrating  the  problem 
in  this  case  is  given  in  Appendix  B.)  Definition  3  avoids 
any  inadvertent  information  leakage  because  rejection  is  not 
based  on  the  actual  secret:  if  there  exists  any  secret  such 
that  a  possible  output  would  reveal  too  much,  the  query  is 
rejected.  Definition  3  resembles,  but  is  stronger  than,  min- 
entropy,  as  the  security  decision  is  based  on  the  most  likely 
secret  from  the  attacker’s  point  of  view  [20];  further  details 
are  given  in  Section  VIII. 

V.  Belief  revision  via  abstract  interpretation 

Consider  how  we  might  implement  belief  tracking  and 
revision  to  enforce  the  threshold  security  property  given  in 
Definition  3.  A  natural  choice  would  be  to  evaluate  queries 
using  a  probabilistic  programming  language  with  support 
for  conditioning;  examples  are  IBAL  [10],  Probabilistic 
Scheme  [11],  and  several  others  [12],  [13],  [14].  In  these 
languages,  probabilistic  evaluation  is  achieved  by  enumer¬ 
ating  inputs  (sampling).  Probabilities  are  associated  with 
each  input  and  tracked  during  execution.  As  more  inputs  are 
enumerated,  a  more  complete  view  of  the  output  distribution 
emerges.  Unfortunately,  to  get  an  accurate  estimate  of  the 
revised  distribution  following  an  output  observation,  one 
must  enumerate  the  entire  input  space,  which  could  be 
quite  large.  If  insufficient  coverage  is  achieved,  then  the 
threshold  check  in  Definition  3  could  either  be  unsound  or 
excessively  conservative,  depending  in  which  direction  an 
implementation  errs. 

To  avoid  sampling,  we  have  developed  a  new  means  to 
perform  probabilistic  computation  based  on  abstract  inter¬ 
pretation.  In  this  approach,  execution  time  depends  on  the 
complexity  of  the  query  rather  than  the  size  of  the  input 
space.  In  the  next  two  sections,  we  present  two  abstract 
domains.  This  section  presents  the  first,  denoted  P,  where  an 
abstract  element  is  a  single  probabilistic  polyhedron,  which 
is  a  convex  polyhedron  [15]  with  information  about  the 
probabilities  of  its  points.  Because  using  a  single  polyhedron 
will  accumulate  imprecision  after  multiple  queries,  in  our 
implementation  we  actually  use  a  different  domain,  denoted 
Vn  (P),  for  which  an  abstract  element  consists  of  a  set  of 
at  most  n  probabilistic  polyhedra  (whose  construction  is 
inspired  by  powersets  of  polyhedra  [25],  [26]).  This  domain, 
described  in  the  next  section,  allows  us  to  retain  precision 
at  the  cost  of  increased  execution  time.  By  adjusting  n,  the 
user  can  trade  off  efficiency  and  precision. 

A.  Polyhedra 

We  first  review  convex  polyhedra,  a  common  technique 
for  representing  sets  of  program  states.  We  use  the  meta- 
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variables  /3,  pi ,  p2,  etc.  to  denote  linear  inequalities.  We 
write  Jv(0)  to  be  the  set  of  variables  occurring  in  f3\ 
we  also  extend  this  to  sets,  writing  fv({/3i, ...,  /3„})  for 

MP i)  u  ■  •  ■  u MPn). 

Definition  4.  A  convex  polyhedron  C  =  ( B ,  V )  is  a 
set  of  linear  inequalities  B  =  {Pi,  ■  ■  ■ , Pm},  interpreted 
conjunctively,  over  dimensions  V.  We  write  C  for  the  set 
of  all  convex  polyhedra.  A  polyhedron  C  represents  a  set  of 
states,  denoted  7c(Cl),  as  follows,  where  a  \=  P  indicates 
that  the  state  a  satisfies  the  inequality  p. 

7c {{B,  V))  =  {<j  |  domain(a)  =  V  ,  V/3  €  B.  a  \=  P} 

Naturally  we  require  that/v({/3i, . . . ,  Pn})  C  V.  We  write 
fv((B,  y))  to  denote  the  set  of  variables  V  of  a  polyhedron. 

Given  a  state  a  and  an  ordering  on  the  variables  in 
domain(o ),  we  can  view  a  as  a  point  in  an  TV-dimensional 
space,  where  TV  =  \domain(a)\.  The  set  7c (C)  can  then 
be  viewed  as  the  integer- valued  lattice  points  in  an  TV- 
dimensional  polyhedron.  Due  to  this  correspondence,  we  use 
the  words  point  and  state  interchangeably.  We  will  some¬ 
times  write  linear  equalities  x  =  f(y)  as  an  abbreviation  for 
the  pair  of  inequalities  x  <  f(y)  and  x  >  f(y). 

Let  C  =  ( B ,  V).  Convex  polyhedra  support  the  following 
operations. 

•  Polyhedron  size,  or  #(C),  is  the  number  of  integer  points 
in  the  polyhedron,  i.e.,  |tc(C)|.  We  will  always  consider 
bounded  polyhedra  when  determining  their  size,  ensuring 
that  #((7)  is  finite. 

•  Expression  evaluation,  ((B))  C  returns  a  convex  polyhe¬ 
dron  containing  at  least  the  points  in  C  that  satisfy  B. 

•  Expression  count,  CffB  returns  an  upper  bound  on  the 
number  of  integer  points  in  C  that  satisfy  B.  (It  may  be 
more  precise  than  ff(((B))  C).) 

•  Meet,  C i  nc  C-2  is  the  convex  polyhedron  containing  ex¬ 
actly  the  set  of  points  in  the  intersection  of  7c(Ci),  7c(C2). 

•  Join,  Ci  Uc  C2  is  the  smallest  convex  polyhedron 
containing  both  j(Ci)  and  7(02). 

•  Comparison,  C\  Lc  C2  is  a  partial  order  whereby 
Ci  Ec  C2  if  and  only  if  7(Ci)  C  7 (C2). 

•  Affine  transform,  C  [x  — >  E\,  where  x  G/v;(C),  computes 
an  affine  transformation  of  C.  This  scales  the  dimension 
corresponding  to  x  by  the  coefficient  of  x  in  E  and  shifts  the 
polyhedron.  For  example,  ({x  <  y,  y  =  2z},  V)  [y  — >  z  +  y] 
evaluates  to  ({x  <  y  —  z,y  —  z  =  2 z},  V). 

•  Forget,  ix(C),  projects  away  x.  That  is,  f X(C)  = 

{ie}(C).  where  7r v(C)  is  a  polyhedron  C'  such  that 
7C (C')  =  {a\a,€  7c(C)  A  a  =  o'  f  V}.  So  C'  =  f X{C) 
implies  x  ^  fv(C'). 

We  write  isempty(C)  iff  7c (C)  =  0. 

B.  Probabilistic  Polyhedra 

We  take  this  standard  representation  of  sets  of  program 
states  and  extend  it  to  a  representation  for  sets  of  distribu¬ 


tions  over  program  states.  We  define  probabilistic  polyhedra, 
the  core  element  of  our  abstract  domain,  as  follows. 

Definition  5.  A  probabilistic  polyhedron  P  is  a  tuple 
(C,  smin,  smax,  pmin,  pmax,  mmin,  rnmax).  We  write  P  for  the 
set  of  probabilistic  polyhedra.  The  quantities  smin  and  smax 
are  lower  and  upper  bounds  on  the  number  of  support  points 
in  the  polyhedron  C.  The  quantities  pmin  and  pmax  are  lower 
and  upper  bounds  on  the  probability  mass  per  support  point. 
The  mmin  and  mmax  components  give  bounds  on  the  total 
probability  mass.  Thus  P  represents  the  set  of  distributions 
7p(P)  defined  below. 

71p(P)  =  {<5  I  supported)  C  7c(C)  A 

smin  <  \support{8)\  <  smax  A 

rnmin  <  \\6\\  <  rnmaxA 

Vcr  G  support (5).  pmin  <  5(a)  <  pmax} 

We  will  write  fv(P)  ==  fv(C)  to  denote  the  set  of  variables 
used  in  the  probabilistic  polyhedron. 

Note  the  set  jp(P)  is  singleton  exactly  when  smm  = 

smax  =  and  prnin  =  pmax^  an(J  mmin  =  mmax  jn  such 

a  case  7p(P)  is  the  uniform  distribution  where  each  state  in 
7c(C)  has  probability  pmm.  Distributions  represented  by  a 
probabilistic  polyhedron  are  not  necessarily  normalized  (as 
was  true  in  Section  III-B).  In  general,  there  is  a  relationship 
between  pmm ,  smm ,  and  mmin,  in  that  mmin  >  pmin  .  smin 
(and  mmax  <  pnlax  •  sma'X)5  and  the  combination  of  the  three 
can  yield  more  information  than  any  two  in  isolation. 

Our  convention  will  be  to  use  Ci,  s™m,  s™ax,  etc.  for  the 
components  associated  with  probabilistic  polyhedron  Pi  and 
to  use  subscripts  to  name  different  probabilistic  polyhedra. 

Distributions  are  ordered  point-wise  [9].  That  is,  <5i  <  h2 
if  and  only  if  Vcr.  5i(a)  <  62(a).  For  our  abstract  domain, 
we  say  that  Pi  LP  P2  if  and  only  if  V<5-|  G  7p(Pl).  3< 52  G 
7p(P2).  Si  <  82.  Testing  Pi  Lp  P2  mechanically  is  non¬ 
trivial,  but  is  unnecessary  in  our  semantics.  Rather,  we 
need  to  test  whether  a  distribution  represents  only  the  zero 
distribution  Onist  =  Acr.O  in  order  to  see  that  a  fixed  point  for 
evaluating  ((while  B  do  S ))  P  has  been  reached.  Intuitively, 
no  further  iterations  of  the  loop  need  to  be  considered  once 
the  probability  mass  flowing  into  the  tr*  iteration  is  zero. 
This  condition  can  be  detected  as  follows: 
iszero(P)  =f 

smin  =  smax  =  0  A  mmin  =  0  <  mmax 

V  mnlin  =  mmax  =  0  A  smin  =  0  <  smax 

V  isempty(C)  A  smin  =  0  <  smax  A  mmin  =  0  <  mmax 

V  pmin  =  pmax  =  0  A  smin  =  0  <  smax  A  mmin  =  0  <  mmax 

If  iszero(P)  holds,  it  is  the  case  that  7p(P)  =  {Onist}-  Note 
that  having  a  more  conservative  definition  of  this  function 
(which  holds  for  fewer  probabilistic  polyhedra)  would  be 
reasonable  since  it  would  simply  mean  our  analysis  would 
terminate  less  often  than  it  could,  with  no  effect  on  security. 
More  details  are  given  in  Appendix  D. 
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In  a  standard  abstract  domain,  termination  of  the  fixed 
point  computation  for  loops  is  often  ensured  by  use  of  a 
widening  operator.  This  allows  abstract  fixed  points  to  be 
computed  in  fewer  iterations  and  also  permits  analysis  of 
loops  that  may  not  terminate.  In  our  setting,  non-termination 
may  reveal  information  about  secret  values.  As  such,  we 
would  like  to  reject  queries  that  may  be  non-terminating. 

We  enforce  this  by  not  introducing  a  widening  operator. 
Our  abstract  interpretation  then  has  the  property  that  it  will 
not  terminate  if  a  loop  in  the  query  may  be  non-terminating 
(and,  since  it  is  an  over-approximate  analysis,  it  may  also 
fail  to  terminate  even  for  some  terminating  computations). 
We  then  reject  all  queries  for  which  our  analysis  fails  to 
terminate.  Loops  do  not  play  a  major  role  in  any  of  our 
examples,  and  so  this  approach  has  proved  sufficient  so  far. 
We  leave  for  future  work  the  development  of  a  widening 
operator  that  soundly  accounts  for  non-termination  behavior. 

Following  standard  abstract  interpretation  terminology,  we 
will  refer  to  V  (Dist)  (sets  of  distributions)  as  the  concrete 
domain,  P  as  the  abstract  domain,  and  7p  :  P  — >  V  (Dist) 
as  the  concretization  function  for  P. 

C.  Abstract  Semantics  for  P 

To  support  execution  in  the  abstract  domain  just  defined, 
we  need  to  provide  abstract  implementations  of  the  basic 
operations  of  assignment,  conditioning,  addition,  and  scaling 
used  in  the  concrete  semantics  given  in  Figure  3.  We  will 
overload  notation  and  use  the  same  syntax  for  the  abstract 
operators  as  we  did  for  the  concrete  operators. 

As  we  present  each  operation,  we  will  also  state  the 
associated  soundness  theorem  which  shows  that  the  abstract 
operation  is  an  over-approximation  of  the  concrete  operation. 
Proofs  are  given  in  Appendix  D.  The  abstract  program 
semantics  is  then  exactly  the  semantics  from  Figure  3,  but 
making  use  of  the  abstract  operations  defined  here,  rather 
than  the  operations  on  distributions  defined  in  Section  III-B. 
We  will  write  (( S ))  P  to  denote  the  result  of  executing  S 
using  the  abstract  semantics.  The  main  soundness  theorem 
we  obtain  is  the  following. 

Theorem  6.  For  all  P ,  5,  if  6  £  'yp(P)  and  ((S))  P  termi¬ 
nates,  then  [5] 5  terminates  and  [5] <5  €  7p(((>5))  P). 

When  we  say  [S']  S  terminates  (or  ((S))  P  terminates) 
we  mean  that  only  a  finite  number  of  loop  unrollings  are 
required  to  interpret  the  statement  on  a  particular  distribution 
(or  probabilistic  polyhedron).  The  precise  definitions  of 
termination  can  be  found  in  Appendix  D. 

We  now  present  the  abstract  operations. 

1 )  Forget:  We  first  describe  the  abstract  forget  operator 
fy(Pi),  which  is  used  in  implementing  assignment.  When  we 
forget  variable  y,  we  collapse  any  states  that  are  equivalent 
up  to  the  value  of  y  into  a  single  state.  To  do  this  correctly, 
we  must  find  an  upper  bound  h™ax  and  a  lower  bound  h™n 
on  the  number  of  points  that  share  the  same  value  of  other 


Figure  4.  Example  of  a  forget  operation  in  the  abstract  domain  P.  In  this 
case,  hJJ1111  =  1  and  h™ax  =  3.  Note  that  h™ax  is  precise  while  h™111  is 
an  under-approximation.  If  sj1111  =  s™ax  =  9  then  we  have  sJJ1111  =  3, 

0max  a  „min  „  min  i  „  max  „  max  a 

b2  —  P2  —  Pi  ‘  ±»  P2  —  P2  ' 


dimensions  x  (this  may  be  visualized  of  as  the  min  and  max 
height  of  Ci  in  the  y  dimension).  Once  these  are  obtained, 
we  have  that  fy(Pi)  ==  P2  where  the  following  hold  of  P2. 


C2 


—.min 

P2 


—  max 

P2 


0min 

b2 


0max 

b2 


fy(C  1) 

pfn  •  max  {h“in  -  (#(Ci) 

—max  •  /  l.  max  cmax  \ 

Pi  '  111111  \LLy  5  / 

[s“in/h™ax] 

min {#&((?!)),  srx} 


Qmin\ 
bl  )) 


1} 


--.min  —-.min 

m2  — 

— max  max 

m2  — 


Figure  4  gives  an  example  of  a  forget  operation  and 
illustrates  the  quantities  h™ax  and  h™ln.  If  C\  =  (Bi,V\), 
the  upper  bound  h™ax  can  be  found  by  maximizing  y  —  y' 
subject  to  the  constraints  Bi  U  Bi[y'/y\,  where  y'  is  a 
fresh  variable  and  Bi[y'/y]  represents  the  set  of  constraints 
obtained  by  substituting  1/  for  y  in  B\.  As  our  points 
are  integer-valued,  this  is  an  integer  linear  programming 
problem  (and  can  be  solved  by  ILP  solvers).  A  less  precise 
upper  bound  can  be  found  by  simply  taking  the  extent  of 
the  polyhedron  C\  along  y,  which  is  given  by  )f(iVy(C\f). 

For  the  lower  bound,  it  is  always  sound  to  use  h™ln  = 
1,  which  is  what  our  implementation  does.  A  more  precise 
estimate  can  be  obtained  by  finding  the  vertex  with  minimal 
height  along  dimension  y.  Call  this  distance  u.  Since  the 
shape  is  convex,  all  other  points  will  have  y  height  greater 
than  or  equal  to  u.  We  then  find  the  smallest  number  of 
integer  points  that  can  be  covered  by  a  line  segment  of  length 
u.  This  is  given  by  [it]  —  1 .  This  value  can  be  taken  as  h“ln. 

Since  the  forget  operator  is  related  to  projection,  we 
state  soundness  in  terms  of  the  projection  operation  on 
distributions.  Note  that  fv{8)  =  domain{domain{8)),  i.e., 
the  domain  of  states  to  which  5  assigns  probability  mass. 


Lemma  7.  If  5  <E  >(P)  then  S  \  (/\^(A)  —  {2/})  €  >(fy(P)). 


We  can  define  an  abstract  version  of  projection  using  forget: 


Definition  8.  Let  f{Xl,x2,...,xn}(P)  =  f{x2, (f*i(P))- 
Then  P  \  V'  =  f (domain(P)-V')(P)- 


That  is,  in  order  to  project  onto  the  set  of  variables  V' , 
we  forget  all  variables  not  in  V' . 

2)  Assignment:  We  have  two  cases  for  abstract  assign¬ 
ment.  If  x  :=  E  is  invertible,2  the  result  of  the  assignment 
Pi  [ x  — >  E]  is  the  probabilistic  polyhedron  P2  such  that 
Ci  —  C\  [x  —>  E]  and  all  other  components  are  unchanged. 

If  the  assignment  is  not  invertible,  then  information  about 
the  previous  value  of  x  is  lost.  In  this  case,  we  use  the  forget 
operation  to  project  onto  the  other  variables  and  then  add  a 
new  constraint  on  x.  Let  P2  =  fx(Pi)  where  C2  =  (B2,  V2). 
Then  P\  [x  — >  E]  is  the  probabilistic  polyhedron  P3  with 
C3  =  (P2  U  {x  =  E}  ,  V2  U  {x})  and  all  other  components 
as  in  P2. 

Lemma  9.  If  S  £  7p(P)  then  6  [n  — >  E]  G  Jp{P[v  — >  E}). 

The  soundness  of  assignment  relies  on  the  fact  that 
our  language  of  expressions  does  not  include  division.  An 
invariant  of  our  representation  is  that  smax  <  #(C).  When 
E  contains  only  multiplication  and  addition  the  above  rules 
preserve  this  invariant;  an  E  containing  division  would 
violate  it.  Division  would  collapse  multiple  points  to  one 
and  so  could  be  handled  similarly  to  projection. 

3 )  Plus:  To  soundly  compute  the  effect  of  plus  we  need  to 
determine  the  minimum  and  maximum  number  of  points  in 
the  intersection  that  may  be  a  support  point  for  both  Pi  and 
for  P2.  We  refer  to  these  counts  as  the  pessimistic  overlap 
and  optimistic  overlap ,  respectively,  and  define  them  below. 

Definition  10.  Given  two  distributions  61,  S2,  we  refer  to 
the  set  of  states  that  are  in  the  support  of  both  <fi  and  S2  as 
the  overlap  of  <5i ,  5-2.  The  pessimistic  overlap  of  Pi  and  P2, 
denoted  Pi  ©  P2,  is  the  cardinality  of  the  smallest  possible 
overlap  for  any  distributions  <5i  G  7p(Pl)  and  62  £  7p(P2). 
The  optimistic  overlap  Pi  ©  P2  is  the  cardinality  of 
the  largest  possible  overlap.  Formally,  we  define  these  as 
follows.  n3  =f  #(Ci  nc  C2),  rii  =f  #(Cl)  —  713,  and 
n2  =  #(C2)  -  n3.  Then 


Pi  ©  P2  =  max  { (s™in  -  ni)  +  (s^lin  -  n2)  -  n3,  0} 
Pi  ©  P2  =  min  {s?1^,  s^ax,  77.3} 


We  can  now  define  abstract  addition. 

Definition  11.  If  not  iszero(Pi)  and  not  iszero(P2) 
then  Pi  +  P2  is  the  probabilistic  polyhedron  P3  = 


2  See  Appendix  D  for  a  precise  definition  of  invertibility. 


(C3,  sfn 

.sr.Pf.Pr)  defined  as  follows. 

C3 

—  Ci  Uc  C2 

„min 

[  „ min  1  „min 

1  Pi  '  P2 

if  Pi  ©  P2  =  #(C3) 

P3 

(min  {p'|nlrl, p™111 } 

otherwise 

max 

f  nmax  „max 

J  Pi  P2 

if  Pi  ©  P2  >  0 

P3 

(  max  {p™ax,  p2lax} 

otherwise 

0min 

b3 

=  max  {s™111  +  s“ln  — 

Pi  ©  P2,  0} 

0max 

b3 

=  min  {s™ax  +  s“ax  — 

Pi  ©  P2,  #(C3)} 

^min 

m3 

=  m™1"  +  m™in  | 

j^max  _  ^  max  ™max 

Ili^  -  ^^"^1  "T  1II2 

If  iszero(Pi)  then  we  define  Pi  +  P2  as  identical  to  P2; 
if  iszero(P2 ),  the  sum  is  defined  as  identical  to  Pi. 

Lemma  12.  If  5i  £  7p(Pi)  and  S2  £  7p(P2)  then  81  +  82  G 
7p(Pi  +  P2). 

4)  Product:  When  evaluating  the  product  P3  =  Pi  x 
P2,  we  assume  that  the  domains  of  Pi  and  P2  are  disjoint, 
i.e.,  Ci  and  C2  refer  to  disjoint  sets  of  variables.  If  Ci  = 
(Bi,Vi)  and  C2  =  (B2,V2),  then  the  polyhedron  Ci  x 
C2  =  (Pi  U  P2,  Vi  U  V2)  is  the  Cartesian  product  of  Ci 
and  C2  and  contains  all  those  states  a  for  which  a  |  V\  G 
7c(Ci)  and  a  \  V2  G  7c (C2).  Determining  the  remaining 
components  is  straightforward  since  Pi  and  P2  are  disjoint. 
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Ci  x  C2 

„  min 

P3 

_  „min  ^min 

—  Pi  •  P2 

„max  _ 

P3  — 

nmax 

Pi 

nmax 

P2 

0min 

b3 

_  0min  0min 

—  bl  '  b2 

Dmax  _ 

S3  — 

0max 

bl 

0max 

b2 

^min 

HI3 

_  ^min  ^  min 

—  LLi1  •  II12 

^max  _ 

m3  — 

^  max 

^  max 

•  II12 

Lemma  13.  For  all  Pi,  P2  such  thatfifPi)  Pifv(P2)  =  0,  if 
<5i  G  7p(Pi)  and  S2  G  7p(P2)  then  61  x  S2  £  7p(Pi  x  P2). 

In  our  examples  we  often  find  it  useful  to  express  uni¬ 
formly  distributed  data  directly,  rather  than  encoding  it  using 
pif.  In  particular,  consider  extending  statements  S  to  include 
the  statement  form  uniform  x  ni  n2  whose  semantics  is 
to  define  variable  x  as  having  values  uniformly  distributed 
between  n  1  and  n2.  Its  semantics  is  as  follows. 

((uniform  x  n  1  n2))  Pi  =  ix{Pi)  x  P2 

Here,  P2  has  p™  =  p^ax  =  ,  s^in  = 

s“ax  =  n2  -  n i  +  l,  mf?in  =  m^ax  =  1,  and  C2  = 
({x  >  ni,x  <  n2}  ,  {x}). 

We  will  say  that  the  abstract  semantics  correspond  to  the 
concrete  semantics  of  uniform  defined  similarly  as  follows. 

[uniform  x  ni  ?r2](5  =  ( 8  \  ft’(8)  —  {x})  x  S2 

where  52  =  (Act.  if  ni  <  tr(x)  <  n2  then  n2_2i+1  else  0). 

The  soundness  of  the  abstract  semantics  follows  immedi¬ 
ately  from  the  soundness  of  forget  and  product. 
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5)  Conditioning:  Distribution  conditioning  for  proba¬ 
bilistic  polyhedra  serves  the  same  role  as  meet  in  the  classic 
domain  of  polyhedra  in  that  each  is  used  to  perform  ab¬ 
stract  evaluation  of  a  conditional  expression  in  its  respective 
domain. 


Definition  14.  Consider  the  probabilistic  polyhedron  Pi  and 
Boolean  expression  B.  Let  n,  n  be  such  that  n  =  Ci#B  and 
n  =  C\  The  value  n  is  an  over-approximation  of  the 

number  of  points  in  Cj  that  satisfy  the  condition  B  and  n  is 
an  over-approximation  of  the  number  of  points  in  C\  that  do 
not  satisfy  B.  Then  P\  |  B  is  the  probabilistic  polyhedron 
P2  defined  as  follows. 

„min  „min  „min  _  „  „  f^min  — 

P2  — Pi  s2  —  max  | Si  —  n,  0| 

„max  „max  „max  „ .  ■  „  r  „max  „  4 

P2  =Pi  s2  =  min  {s!  ,n| 

m2  =  max  |p2  ■  s2  ,  mj  —  pi  -mm{si  ,  njj 

m2lax  =  min  {p?ax  •  sT*,  mfax  -  pf"  •  max  {sf111  -  n,  0}} 

C2  =  ((B))  Ci 

The  maximal  and  minimal  probability  per  point  are  un¬ 
changed,  as  conditioning  simply  retains  points  from  the 
original  distribution.  To  compute  the  minimal  number  of 
points  in  P2,  we  assume  that  as  many  points  as  possible  from 
Ci  fall  in  the  region  satisfying  ~^B.  The  maximal  number 
of  points  is  obtained  by  assuming  that  a  maximal  number 
of  points  fall  within  the  region  satisfying  B. 

The  total  mass  calculations  are  more  complicated.  There 
are  two  possible  approaches  to  computing  m™ln  and  m“ax. 
The  bound  m™n  can  never  be  less  than  p™ln  •  s™ln,  and 
so  we  can  always  safely  choose  this  as  the  value  of  m™ln. 
Similarly,  we  can  always  choose  p2nax  •  s™ax  as  the  value 
of  m™ax.  However,  if  m™“  and  m™  give  good  bounds 
on  total  mass  (i.e.,  m™m  is  much  higher  than  p™ln  •  sfln 
and  dually  for  m™ax),  then  it  can  be  advantageous  to  reason 
starting  from  these  bounds. 

We  can  obtain  a  sound  value  for  m“m  by  considering 
the  case  where  a  maximal  amount  of  mass  from  Ci  fails  to 
satisfy  B.  To  do  this,  we  compute  n  =  Ci#^B,  which 
provides  an  over-approximation  of  the  number  of  points 
within  Ci  but  outside  the  area  satisfying  B.  We  bound  n 
by  s™ax  and  then  assign  each  of  these  points  maximal  mass 
p'i"ax,  and  subtract  this  from  my1"1,  the  previous  lower  bound 
on  total  mass. 

By  similar  reasoning,  we  can  compute  m™ax  by  assuming 
a  minimal  amount  of  mass  m  is  removed  by  conditioning, 
and  subtracting  m  from  m^nax.  This  m  is  given  by  consider¬ 
ing  an  under-approximation  of  the  number  of  points  falling 
outside  the  area  of  overlap  between  Ci  and  B  and  assigning 
each  point  minimal  mass  as  given  by  p™ln.  This  m  is  given 
by  max(sylln  —  n,  0). 

Figure  5  demonstrates  the  components  that  affect  the 
conditioning  operation.  The  figure  depicts  the  integer-valued 
points  present  in  two  polyhedra — one  representing  Ci  and 
the  other  representing  B  (shaded).  As  the  set  of  points  in  Cl 
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Figure  5.  Example  of  distribution  conditioning  in  the  abstract  domain  P. 


satisfying  B  is  convex,  this  region  is  precisely  represented  by 
((B))  Ci.  By  contrast,  the  set  of  points  in  Ci  that  satisfy  ~^B 
is  not  convex,  and  thus  {(— H))  Cl  is  an  over-approximation. 
The  icons  beside  the  main  image  indicate  which  shapes 
correspond  to  which  components  and  the  numbers  within 
the  icons  give  the  total  count  of  points  within  those  shapes. 
Suppose  the  components  of  Pi  are  as  follows. 

s“in  =  19  pfin  =  0.01  mfn  =  0.85 

s“ax  =  20  pynax  =  0.05  m™^  =  0.9 

Then  n  =  4  and  n  =  16.  Note  that  we  have  set  n  to  be  the 

number  of  points  in  the  non-shaded  region  of  Figure  5.  This 
is  more  precise  than  the  count  given  by  #(((P))  C),  which 
would  yield  18.  This  demonstrates  why  it  is  worthwhile  to 
have  a  separate  operation  for  counting  points  satisfying  a 
boolean  expression.  These  values  of  n  and  n  give  us  the 
following  for  the  first  four  numeric  components  of  P2. 

sf*in  =  max(19  -  16, 0)  =  3  p^in  =  0.01 
gmax  =  min(20, 4)  =  4  p”ax  =  0.05 

For  the  m™m  and  m™ax,  we  have  the  following  for  the 

,!  1  r  1  t  ..  1  ,  min/max  ,  min/max 

method  or  calculation  based  on  p2  and  s2 

m“in  =  0.01  •  3  =  0.03  m“ax  =  0.05  •  4  =  0.2 

For  the  method  of  computation  based  on  m™m/maxi  we  have 

m“in  =  0.85  -  0.05  •  16  =  0.05 
m“ax  =  0.9  -  0.01  •  (19  -  4)  =  0.75 

In  this  case,  the  calculation  based  on  subtracting  from 
total  mass  provides  a  tighter  estimate  for  m™ln,  while  the 
method  based  on  multiplying  p“ax  and  s™ax  is  better  for 

^max 

m2 

Lemma  15.  If  5  e  7p(P)  then  6\B  £  jp(P  \  B). 


6)  Scalar  Product:  The  scalar  product  is  straightforward, 
as  it  just  scales  the  mass  per  point  and  total  mass. 


Definition  16.  Given  a  scalar  p  in  [0, 1],  we  write  p  ■  Pi  for 
the  probabilistic  polyhedron  P2  specified  below. 


C1mm  _  0min 

b2  — 

cmax  _  cmax 

b2  — 

=  p  •  mj1111 

™max  _  „  ™  max 

m2  —  P  ' 

Lemma  17.  If  8\  £  jp(Pi] 


nmin  _  „  .  nmin 

P2  —  P  Pi 

„  max  _  „  nmax 

P2  ~  P  ‘  Pi 
C2  =  Cl 


then  p  ■  S 1  £  7 p(p  ■  Pi). 
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7)  Normalization:  If  a  probabilistic  polyhedron  P  has 
mmin  =  1  and  mmax  =  1  then  it  represents  a  normal¬ 
ized  distribution.  We  define  below  an  abstract  counterpart 
to  distribution  normalization,  capable  of  transforming  an 
arbitrary  probabilistic  polyhedron  into  one  containing  only 
normalized  distributions. 


Definition  18.  Whenever  m“ln  >  0,  we  write  normal  (Pi ) 
for  the  probabilistic  polyhedron  P2  specified  below. 


—.min 

P2 

—max 

P2 


_  —min  /™ma] 

—  px  /m1 

_  —max  /--.mil 

—  p:  /m1 

=  m19nax  =  1 


51 

.,max 


c2  =  Ci 


When  m"11"  =  0,  we  set  p™ax  =  1.  Note  that  if  Pi  is  the 
zero  distribution  then  normal(Pi)  is  not  defined. 


Lemma  19.  If  Si  £  7p(Pi)  and  normal(Si)  is  defined,  then 
normal(5i)  £  "fp(normal(Pi)). 


D.  Policy  Evaluation 

Here  we  show  how  to  implement  the  threshold  test  given 
as  Definition  3  using  probabilistic  polyhedra.  To  make  the 
definition  simpler,  let  us  first  introduce  a  bit  of  notation. 

Notation  20.  If  P  is  a  probabilistic  polyhedron  over  vari¬ 
ables  V,  and  <7  is  a  state  over  variables  V  C  V ,  then 
P  \  a  =  P  \  B  where  B  =  /\x€V,  x  =  <j(x). 

Definition  21.  Given  some  probabilistic  polyhedron  Pi  and 
statement  S  where  (( S ))  Pi  terminates,  let  P2  =  ((S))  Pi  and 
P3  =  P2  f  L.  If,  for  every  07,  £  7c  (C3)  with  ->iszero(P2  \ 
07),  we  have  P4  =  normal((P2  |  07)  \  H)  with  P4iax  <  t, 
then  we  write  tsecuret(S,  Pi). 

The  computation  of  P3  involves  only  abstract  interpre¬ 
tation  and  projection,  which  are  computable  using  the  op¬ 
erations  defined  previously  in  this  section.  If  we  have  a 
small  number  of  outputs  (as  for  the  binary  outputs  consid¬ 
ered  in  our  examples),  we  can  enumerate  them  and  check 
~^iszero(P2  \  <jl)  for  each  output  07,.  When  this  holds 
(that  is,  the  output  is  feasible),  we  compute  P4,  which  again 
simply  involves  the  abstract  operations  defined  previously. 
The  final  threshold  check  is  then  performed  by  comparing 
pmax  t0  probability  threshold  t. 

Now  we  state  the  main  soundness  theorem  for  abstract 
interpretation  using  probabilistic  polyhedra.  This  theorem 
states  that  the  abstract  interpretation  just  described  can  be 
used  to  soundly  determine  whether  to  accept  a  query. 

Theorem  22.  Let  S  be  an  attacker’s  initial  belief.  If  5  £ 
7p(P-i  )  and  tsecuret(S,  Pi),  then  S  is  threshold  secure  for 
threshold  t  when  evaluated  with  initial  belief  S. 


VI.  POWERSET  OF  PROBABILISTIC  POLYHEDRA 

This  section  presents  the  Vn  (P)  domain,  an  extension  of 
the  P  domain  that  abstractly  represents  a  set  of  distributions 
as  at  most  n  probabilistic  polyhedra,  elements  of  P. 


Definition  23.  A  probabilistic  ( polyhedral )  set  A  is  a 
set  of  probabilistic  polyhedra,  or  {Pi}  with  each  Pi  over 
the  same  variables.  We  write  Vn  (P)  for  the  domain  of 
probabilistic  polyhedral  powersets  composed  of  no  more 
than  n  probabilistic  polyhedra. 

Each  probabilistic  polyhedron  P  is  interpreted  disjunc¬ 
tively:  it  characterizes  one  of  many  possible  distributions. 
The  probabilistic  polyhedral  set  is  interpreted  additively.  To 
define  this  idea  precisely,  we  first  define  a  lifting  of  +  to 
sets  of  distributions.  Let  D 1,  D2  be  two  sets  of  distributions. 
We  then  define  addition  as  follows. 

Di  +  D-2  =  {<5i  +  62  |  #1  £  Di  A  S2  £  D2} 

This  operation  is  commutative  and  associative  and  thus  we 
can  use  for  summations  without  ambiguity  as  to  order 
of  operations.  The  concretization  function  for  V„  (P)  is  then 
defined  as: 

7p„(p)(A)  =  ^2  7p (P) 

Pe  A 

We  can  characterize  the  condition  of  A  containing  only 
the  zero  distribution,  written  iszero} A),  via  the  condition 
that  all  of  the  member  probabilistic  polyhedra  are  zero. 

iszero{ A)  =f  iszero{P) 

PeA 

A.  Abstract  Semantics  for  Vn  (P) 

With  a  few  exceptions,  the  abstract  implementations  of 
the  basic  operations  for  the  powerset  domain  are  extensions 
of  operations  defined  on  the  base  probabilistic  polyhedra 
domain. 

Theorem  24.  For  all  S,  S,  A,  if  S  £  7p„(p)(A)  and 
((S'))  A  terminates,  then  [S]<5  terminates  and  [S]<5  £ 
7p„(p )(((£))  A). 

Proof  of  this  theorem  is  given  in  Appendix  E. 

Definition  25.  The  powerset  simplification  transforms  a 
set  containing  potentially  more  than  n  elements  into  one 
containing  no  more  than  n,  for  n  >  1.  The  simplest  approach 
involves  repeated  use  of  abstract  plus  in  the  base  domain  P. 

rpim  I  def  j  {Pi\i= 1  if  m  <  n 

Li  °i= iJn  l  u  {Pm-1  +  Pm}\n  otherwise 

Lemma  26.  7p„(p)(A)  C  7prl(p)(|_AJm)  where  m  <  n. 

Note  that  the  order  in  which  individual  probabilistic 
polyhedra  are  simplified  has  no  effect  on  soundness  but  may 
impact  the  precision  of  the  resulting  abstraction. 

Many  of  the  operations  and  lemmas  for  the  powerset 
domain  are  simple  liftings  of  the  corresponding  operations 
and  lemmas  for  single  probabilistic  polyhedra.  For  these 
operations  (operations  1-5  given  below),  we  simply  list  the 
definition. 

1)  Forget:  fy(A)  dAf  {fy(P)  |  P  £  A} 
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2)  Project:  A  \  V  =  {P  \  V  \  P  £  A} 

3)  Conditioning:  A  \  B  =  {P  \  B  \  P  £  A} 

4)  Assignment:  A  [x  — >  E\  =f  {P[x  — >  E\  \  P  £  A} 

5 )  Scalar  product:  p  ■  A  =  {p  ■  P  \  P  £  A} 

6)  Product:  The  product  operation  is  only  required  for 
the  special  uniform  statement  and  only  applies  to  the  product 
of  a  probabilistic  set  with  a  single  probabilistic  polyhedron. 
A  x  P'  =f  {P  x  P1  \  P  £  A}  (where  we  assume  that 
fv{ A)  CP'  =  0). 

7)  Plus:  The  abstract  plus  operation  involves  simplifying 
the  combined  contributions  from  two  sets  into  one  bounded 
set:  Ax  +  A2  =  |Ai  U  A2J„,  whenever  ->iszero(Ai)  and 
->zs2ero(A2).  Alternatively,  if  iszero( Ai)  (or  iszero( A2)) 
then  Ai  +  A2  is  defined  to  be  identical  to  A2  (or  Ai). 

8)  Normalization:  Since  in  the  Vn  (P)  domain,  the 
over(under)  approximation  of  the  total  mass  is  not  contained 
in  any  single  probabilistic  polyhedron,  the  normalization 
must  scale  each  component  of  a  set  by  the  overall  total.  The 
minimum  (maximum)  mass  of  a  probabilistic  polyhedra  set 
A  =  {Pi, . . . ,  P„}  is  defined  as  follows. 

Mmin  (  a  )  =  £!*=1  m“in  |  Mmax(A)  =  £"=1  m“ax 


Definition  27.  The  scaling  of  a  probabilistic  polyhedra  Pi 
by  minimal  total  mass  to  and  maximal  total  mass  to,  written 
normal(P)  (to,  to)  is  the  probabilistic  polyhedron  P2  defined 
as  follows  whenever  to  >  0. 


„  min 

P2 

=  p™ln/TO 

„max 

P2 

=  p  “'“/to 

min 

m2 

_  mmin  jYfi 

^  max 
m2 

=  m"lax/TO 

0min 

b2 

0max 

b2 

c2 


_  cmin 

— 

_  0max 

—  h1 

=  Ci 


Whenever  to  =  0  the  resulting  P2  is  defined  as  above  but 
with  p!fax  =  1  and  m(fax  =  1. 


Normalizing  a  set  of  probabilistic  polyhedra  can  be  de¬ 
fined  as  follows 


normal(A)  =  {norrnal(P)(Mmin(A),Mmax(A))  |PgA} 
B.  Policy  Evaluation 

Determining  the  bound  on  the  probability  of  any  state 
represented  by  a  single  probabilistic  polyhedron  is  as  simple 
as  checking  the  pmax  value  in  the  normalized  version  of 
the  probabilistic  polyhedron.  In  the  domain  of  probabilistic 
polyhedron  sets,  however,  the  situation  is  more  complex,  as 
polyhedra  may  overlap  and  thus  a  state’s  probability  could 
involve  multiple  probabilistic  polyhedra.  A  simple  estimate 
of  the  bound  can  be  computed  by  abstractly  adding  all  the 
probabilistic  polyhedra  in  the  set,  and  using  the  pmax  value 
of  the  result. 


Lemma  28.  If  5  £  7p„(p)(A)  and  Pi  =  J2Pe/±P  then 
maxo-  <5(cr)  <  p“ax. 

This  approach  has  an  unfortunate  tendency  to  increase  the 
probability  bound  determined  as  one  increases  the  bound 


on  the  number  of  probabilistic  polyhedra  allowed.  A  more 
complicated  method,  which  is  used  in  our  implementation, 
computes  a  partition  of  the  polyhedra  in  the  set  into  another 
set  of  disjoint  polyhedra  and  determines  the  maximum 
probable  point  among  the  representatives  of  each  region  in 
the  partition.  In  order  to  present  this  method  precisely  we 
begin  with  some  definitions. 


Definition  29.  The  (maximum)  probability  of  a  state  a 
according  to  a  probabilistic  polyhedron  Pi,  written  P™ax  (cr), 
is  p™ax  if  p1  £  7c (Ci)  and  0  otherwise. 


pr  (<t) 


p“ax  if  a  e  7c(Ci) 

0  otherwise 


Likewise  the  (maximum)  probability  of  cr  according  to  a 
probabilistic  polyhedra  set  A  =  {Pi},  written  Amax  (cr),  is 
defined  as  follows. 


Amax  (a)  =  prx  M 

i 

A  mere  application  of  the  various  definitions  allows  one 
to  conclude  the  following  remark. 

Remark  30.  If  8  £  7p„(p)(A)  then  S(a)  <  Amax  (cr)  and 
therefore  max^  S(a)  <  maxff  Amax  (CT)  for  every  cr. 

Taking  advantage  of  the  domain,  we  will  produce  a 
set  of  representative  points  {cr.;}  with  max,;  Amax  (cr,)  = 
maxg.  Amax  (cr).  To  do  this,  we  first  need  to  define  a  linear 
partition. 

Definition  31.  A  poly  partition  of  a  set  of  polyhedra  {Pi} 
is  another  set  of  polyhedra  { L, },  usually  of  larger  size,  with 
the  following  properties. 

1)  7c {Li)  n  7c(£j)  =  0  for  every  i  ^  j. 

2)  U;7c(Li)  =  Ui7c(T>,) 

3)  For  every  i,j,  either  7c (L,:)  C  7 c(Pj)  or  7c (Li)  n 
7C  (Pj)  =  0- 

Any  set  {cr.;},  with  cr;  £  jc(Li)  for  every  i,  will  be  called 
a  representative  set  of  the  partition. 

We  can  now  determine  the  maximal  probability  using 
only  representative  points,  one  from  each  piece  of  the  poly 
partition. 

Lemma  32.  maxCTgfl  Amax  (a)  =  maxa  Amclx  (a)  where  C 
is  a  poly  partition  of  A  and  R  is  a  representative  set  of  C. 

Note  that  the  set  of  representatives  R  is  not  unique  and 
the  lemma  holds  for  any  such  set. 

We  will  write  maxpp( A)  for  maxff  Amax  (cr)  to  make 
explicit  the  method  with  which  this  value  can  be  computed 
according  to  the  lemma  above. 

Notation  33.  If  A  is  a  probabilistic  polyhedron  set  over 
variables  V,  and  cr  is  a  state  over  variables  V'  C  V,  then 
A  |  cr  =  A  |  B  where  B  =  /\xeV,  x  =  cr(a;). 
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Figure  6.  Query  evaluation  comparison 


Definition  34.  Given  some  probabilistic  polyhedron  Ai  and 
statement  S  where  (( S ))  Ai  terminates,  let  A2  =  ((S)) 
and  A3  =  A2  \  L  =  {P'}.  If  for  every  aL  <E  7 7>(c)  ({£*}) 
with  ^is2ero(A2  |  07J  we  have  A4  =  normal((A2  |  07)  f 
H)  and  maxpp  (A4)  <  t,  then  we  write  tsecuret(S,  Ai). 

Below  we  state  the  main  soundness  theorem  for  abstract 
interpretation  using  probabilistic  polyhedron  sets.  This  theo¬ 
rem  states  that  the  abstract  interpretation  just  described  can 
be  used  to  soundly  determine  whether  to  accept  a  query. 

Theorem  35.  Let  S  be  an  attacker’s  initial  belief.  If  6  7 
7pti(p)(A)  and  tsecuret(S,  A),  then  S  is  threshold  secure 
for  threshold  t  when  evaluated  with  initial  belief  5. 

VII.  Implementation  and  experiments 

We  have  implemented  an  interpreter  for  the  core  language 
based  on  the  probabilistic  polyhedra  powerset  domain.  The 
base  manipulations  of  polyhedra  are  done  using  the  Parma 
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Polyhedra  Library  [17],  Size  calculations  are  done  using  the 
LattE  lattice  point  counter  [18].  LattE  is  also  used  for  the 
integer  linear  programming  problem  involved  in  the  abstract 
forget  operation.  The  interpreter  itself  is  written  in  OCaml. 
We  conducted  several  experiments  on  a  Mac  Pro  with  two 
2.26  GHz  quad-core  Xeon  processors  using  16  GB  of  RAM 
and  running  OS  X  vlO.6.7.  While  many  of  the  abstract  oper¬ 
ations  distribute  over  the  set  of  probabilistic  polyhedra  and 
thus  could  be  parallelized,  our  implementation  is  currently 
single-threaded. 

Figure  6(a)  illustrates  the  result  of  running  the  query 
given  in  Example  1  (Section  II)  using  our  implementation 
and  one  using  Probabilistic  Scheme  [11],  which  is  capable 
of  sound  probability  estimation  after  partial  enumeration. 
Each  x  plots  prob-scheme’s  maximum  probability  value 
(the  y  axis) — that  is,  the  probability  it  assigns  to  the  most 
likely  secret  state — when  given  a  varying  amount  of  time 
for  sampling  (the  x  axis).  We  can  see  the  precision  improves 
steadily  until  it  reaches  the  exact  value  of  1/259  at  around 
17  seconds.  Each  +  plots  our  implementation’s  maximum 
probability  value  when  given  an  increasing  number  of  prob¬ 
abilistic  polyhedra;  with  a  polyhedral  bound  of  2  (or  more), 
we  obtain  the  exact  value  in  less  than  3  seconds.  The  timing 
measurements  are  taken  to  be  the  medians  of  12  runs. 
The  advantage  of  our  approach  is  more  evident  in  Figure 
6(b)  where  we  use  the  same  program  but  allow  byear  to 
span  1910  to  2010  rather  than  1956  to  1992.  In  this  case 
prob-scheme  makes  little  progress  even  after  a  minute,  and 
eventually  runs  out  of  memory.  Our  approach,  however,  is 
unaffected  by  this  larger  state  space  and  produces  the  exact 
maximum  belief  in  around  3  seconds  when  using  only  2 
probabilistic  polyhedra. 

Figure  6(c)  shows  the  result  of  our  implementation  as¬ 
sessing  the  special  query  (Example  2)  with  initial  belief 
matching  that  following  the  first  birthday  query.  Each  plot¬ 
ted  point  is  the  number  of  polyhedra  allowed.  The  result 
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0.6  (1) 

l.i  0) 
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0.14286 

0.14286 

0.14286 

0.14286 

0.14286 

0.14286 
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0.14286 

0.14286 

0.14286 

0.14286 

0.14286 

0.14286 

0.14286 

travel 

214.8 

21.4 

34.9 

33.7 

59.6 

46.8 

74.8 

62.0 

77.5 

72.9 

139.2 

133.2 

149.4 

163.3 

170.2 

163.0 

235.5 

7.1  (0) 

0.8  (3) 

4.1  (2) 

6.9  (0) 

8.8  (0) 

6.6  (0) 

1.8  (4) 

3.6  (3) 

11.4  (0) 

5.7  (2) 

5.9  (3) 

8.0  (3) 

22.5  (0) 

14.0  (2)  6.3  (3) 

15.1  (2) 

28.1  (2) 

l 

1 

1 

1 

1 

0.01111 

0.01111 

0.00556 

0.00556 

0.00139 

0.00123 

0.00101 

5.05e-4 

5.05e-4 

5.05e-4 

5.05e-4 

5.05e-4 

Table  I 

Query  evaluation  benchmarks 


demonstrates  that  more  complex  queries,  specifically  ones 
with  many  disjunctions  in  their  conditionals,  not  only  slow 
our  approach,  but  also  reduce  the  precision  of  the  maximum 
probability  value.  The  example  requires  36  polyhedra  for 
exact  calculations  though  as  little  as  3  produce  probabili¬ 
ties  near  exact.  Note  that  the  precision  does  not  increase 
monotonically  with  the  number  of  polyhedra — in  some  cases 
more  polyhedra  leads  to  a  less  precise  result.  We  conjecture 
that  the  occasional  worsening  of  the  precision  with  increase 
in  the  number  of  allowable  polyhedra  is  due  to  an  overly 
simple  means  of  deciding  which  polyhedra  to  merge  when 
performing  abstract  simplification;  we  plan  to  investigate  this 
issue  in  future  work. 

Table  I  tabulates  details  for  the  example  programs  along 
three  other  queries  we  developed  based  on  advertising 
scenarios;  these  queries  are  described  in  the  Appendix  C. 
In  each  box  is  the  wall  clock  time  for  processing  (median 
of  12  runs),  the  running  time’s  semi-interquartile  range 
(SIQR),  the  number  of  outliers,  which  are  defined  to  be 
the  points  3  x  SIQR  below  the  first  quartile  or  above  the 
third,  and  the  max  belief  computed  (smaller  being  more 
accurate).  Obvious  trends  are  that  running  time  goes  up 
and  max  belief  goes  down  as  the  number  of  polyhedra 
increase,  by  and  large.  There  are  exceptions  to  running  time 
trend,  and  most  are  close  to  the  SIQR  and  so  possibly  not 
statistically  significant.  The  most  striking  exception  is  the 


running  time  for  poly-size  9  of  the  “pizza”  query.  This 
extreme  outlier  is  due  to  a  single  invocation  of  LattE  on 
the  largest  set  of  constraints  among  all  the  benchmarks 
performed  in  the  table.  We  have  no  good  explanation  of 
how  this  complex  polyhedron  arose.  The  only  exceptions  to 
monotonic  decrease  in  max  belief  are  the  “special  queries”, 
as  already  discussed. 

Investigating  the  running  time  results  further,  we  discov¬ 
ered  that  for  nearly  all  benchmarks,  95%  or  more  of  the 
running  time  is  spent  in  the  LattE  counting  tool.  The  LattE 
tool  exhibits  super-exponential  running  time  in  terms  of  the 
number  of  constraints  (see  Figure  7)  over  the  polyhedra  that 
occur  when  evaluating  the  various  queries  in  Table  I.  As 
such,  overall  running  time  is  susceptible  to  the  complexity  of 
the  polyhedra  involved,  even  when  they  are  few  in  number. 
The  merging  operation,  while  used  to  keep  the  number  of 
probabilistic  polyhedra  below  the  required  bound,  also  tends 
to  produce  more  complex  polyhedra.  These  observations 
suggest  a  great  deal  of  performance  improvement  can  be 
gained  by  simplifying  the  polyhedra  if  they  become  too 
complex. 

VIII.  Discussion  and  related  work 

Prior  work  aimed  at  controlling  access  to  users’  private 
data  has  focused  on  access  control  policies.  For  example. 
Persona  [6]  users  can  store  personal  data  on  distributed 
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storage  servers  that  use  attribute-based  encryption;  only 
those  parties  that  have  the  attribute  keys  for  particular  data 
items  may  see  them.  Our  approach  relaxes  the  access  control 
model  to  offer  more  fine-grained  information  release  policies 
by  directly  modeling  an  attacker’s  belief. 

Others  have  considered  how  an  adversary’s  knowledge 
of  private  data  might  be  informed  by  a  program’s  output. 
Clark,  Hunt,  and  Malacaria  [27]  define  a  static  analysis 
that  bounds  the  secret  information  a  straight-line  program 
can  leak  in  terms  of  equivalence  relations  between  the 
inputs  and  outputs.  Backes  et  al.  [21]  automate  the  syn¬ 
thesis  of  such  equivalence  relations  and  quantify  leakage 
by  computing  the  exact  size  of  equivalence  classes.  Kopf 
and  Rybalchenko  [22]  extend  this  approach,  improving  its 
scalability  by  using  sampling  to  identify  equivalence  classes 
and  using  under-  and  over-approximation  to  obtain  bounds 
on  their  size.  Mu  and  Clark  [28]  present  a  similar  analysis 
that  uses  over-approximation  only.  In  all  cases,  the  inferred 
equivalence  classes  can  be  used  to  compute  entropy-based 
metrics  of  information  leakage. 

We  differ  from  this  work  in  two  main  ways.  First,  we 
implement  a  different  security  criterion.  The  most  closely 
related  metric  is  vulnerability  V  as  proposed  by  Smith  [20], 
which  can  be  defined  using  our  notation  as  follows;3 

Definition  36.  Let  S'  =  [S']  <5,  where  S  is  the  model  of  the 
querier’s  initial  belief,  and  let  5x  =  normal(c)  [  X).  Then 
query  S  is  vulnerability  threshold  secure  iff  for 

V=  5'L(aL)  ■  max  (5'\aL)H(oH) 

<r  L£support(5  L) 

we  have  V  <  t  for  some  threshold  t. 

The  above  definition  is  an  expectation  over  all  possible 
outputs  ol,  so  unlikely  outputs  have  less  influence.  Our  no¬ 
tion  of  threshold  security  (Definition  3)  is  stronger  because 
it  considers  each  output  individually:  if  any  output,  however 
unlikely,  would  increase  knowledge  beyond  the  threshold, 
the  query  would  be  rejected.  For  example,  recall  the  query 
from  Example  1  where  the  secret  data  bday  is  (assumed 
by  the  querier  to  be)  uniformly  distributed;  call  this  query 
Q  t .  According  to  Definition  36,  the  minimum  acceptable 
threshold  t  >  V  =  2/365  ss  0.005,  whereas  according  to 
Definition  3,  the  minimum  threshold  is  t  >  1/7  w  0.143 
which  corresponds  the  equivalence  class  260  <  bday  <  267. 

The  other  main  difference  is  that  we  keep  an  on-line 
model  of  knowledge  according  to  prior,  actual  query  results, 
which  increases  our  precision.  To  see  the  benefit  consider 
performing  query  Qi  followed  by  a  query  Qi  which  uses 
the  code  from  Example  1  but  has  today  =  265.  With  our 
system  and  bday  =  270  the  answer  to  Q i  is  False  and 
with  the  revised  belief  the  query  Q 2  will  be  accepted  as 
below  threshold  t(j  =  0.2.  If  instead  we  had  to  model  this 

3 Smith  actually  proposes  min  entropy ,  which  is  —log  V. 


pair  of  queries  statically  they  would  be  rejected  because 
(under  the  assumption  of  uniformity)  the  pair  of  outputs 
True.True  is  possible  and  implies  bday  £  {265,266}  which 
would  require  tg  >  0.5.  Our  approach  also  inherits  from 
the  belief-based  approach  the  ability  to  model  a  querier 
who  is  misinformed  or  incorrect,  which  can  arise  following 
the  result  of  a  probabilistic  query  (more  on  this  below)  or 
because  of  a  change  to  the  secret  data  between  queries  [9]. 
On  the  other  hand,  these  advantages  of  our  approach  come 
at  the  cost  of  maintaining  on-line  belief  models. 

Our  proposed  abstract  domains  P  and  Vn  (P)  are  useful 
beyond  the  application  of  belief-based  threshold  security; 
e.g.,  they  could  be  used  to  model  uncertainty  off-line  (as 
in  the  above  work)  rather  than  beliefs  on-line,  with  the 
advantage  that  they  are  not  limited  to  uniform  distributions 
(as  required  by  [21],  [22]).  Prior  work  on  probabilistic 
abstract  interpretation  is  insufficient  for  this  purpose.  For 
example,  Monniaux  [29]  gives  an  abstract  interpretation 
for  probabilistic  programs  based  on  over-approximating 
probabilities.  That  work  contains  no  treatment  of  distribu¬ 
tion  conditioning  and  normalization,  which  are  crucial  for 
belief-based  information  flow  analysis.  The  use  of  under¬ 
approximations,  needed  to  soundly  handle  normalization,  is 
unique  to  our  approach. 

McCamant  and  Ernst’s  FlowCheck  tool  [19]  measures 
the  information  released  by  a  particular  execution.  However, 
it  measures  information  release  in  terms  of  channel  capacity , 
rather  than  remaining  uncertainty  which  is  more  appropriate 
for  our  setting.  For  example,  FlowCheck  would  report 
a  query  that  tries  to  guess  a  user’s  birthday  leaks  one  bit 
regardless  of  whether  the  guess  was  successful,  whereas 
the  belief-based  model  (and  the  other  models  mentioned 
above)  would  consider  a  failing  guess  to  convey  very  little 
information  (much  less  than  a  bit),  and  a  successful  guess 
conveying  quite  a  lot  (much  more  than  a  bit). 

To  avoid  reasoning  directly  about  an  adversary’s  knowl¬ 
edge,  Dwork  and  colleagues  proposed  differential  pri¬ 
vacy  [24]:  a  differentially  private  query  over  a  database  of 
individuals’  records  is  a  randomized  function  that  produces 
roughly  the  same  answer  whether  a  particular  individual’s 
data  is  in  the  database  or  not.  Thus,  if  the  database  curator 
is  trustworthy,  there  is  little  reason  for  an  individual  to  not 
supply  his  data.  However,  we  prefer  users  to  control  access 
to  their  data  as  they  like,  rather  than  have  to  trust  a  curator. 

In  any  case,  it  is  difficult  to  see  how  to  effectively  adapt 
differential  privacy,  which  was  conceived  for  queries  over 
many  records,  to  queries  over  an  individual’s  record,  as  in 
our  setting.  To  see  why,  consider  the  birthday  query  from 
Example  1.  Bob’s  birthday  being/not  being  in  the  query 
range  influences  the  output  of  the  query  only  by  1  (assuming 
yes/no  is  1/0).  One  could  add  an  appropriate  amount  of 
(Laplacian)  noise  to  the  query  answer  to  hide  what  the  true 
answer  was  and  make  the  query  differentially  private.  How¬ 
ever,  this  noise  would  be  so  large  compared  to  the  original 
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range  {0, 1}  that  the  query  becomes  essentially  useless — 
the  user  would  be  receiving  a  birthday  announcement  most 
days.4  By  contrast,  our  approach  permits  answering  queries 
exactly  if  the  release  of  information  is  below  the  threshold. 
Moreover,  there  is  no  limit  on  the  number  of  queries  as 
long  the  information  release  remains  bounded;  differential 
privacy,  in  general,  must  impose  an  artificial  limit  (termed 
the  privacy  budget )  because  it  does  not  reason  about  the 
information  released. 

Nevertheless,  differential  privacy  is  appealing,  and  it 
would  be  fruitful  to  consider  how  to  apply  its  best  attributes 
to  our  setting.  Rastogi  and  Suciu  [23]  propose  a  property 
called  adversarial  privacy  that  suggests  a  way  forward.  Like 
our  approach,  adversarial  privacy  is  defined  in  terms  of  a 
change  in  attacker  knowledge.  Roughly:  a  query’s  output 
on  any  database  may  increase  an  attacker’s  a  priori  belief 
5(a)  about  any  state  er  by  at  most  e  for  all  5  £  D  for 
some  D  £  'P(Dist).  Rastogi  and  Suciu  show  that,  for  a 
certain  class  D ,  adversarial  privacy  and  differential  privacy 
are  equivalent,  and  by  relaxing  the  choice  of  D  one  can 
smoothly  trade  off  utility  for  privacy.  We  can  take  the 
reverse  tack:  by  modeling  a  (larger)  set  of  beliefs  we  can 
favor  privacy  over  utility.  Our  abstractions  P  and  Vn  (P) 
already  model  sets  of  distributions,  rather  than  a  single 
distribution,  so  it  remains  interesting  future  work  to  exploit 
this  representation  toward  increasing  privacy. 

Another  important  open  question  for  our  work  is  means 
to  handle  collusion.  Following  our  motivating  example  in 
the  Introduction,  the  user’s  privacy  would  be  thwarted  if  he 
shared  only  his  birth  day  with  querier  X  and  only  his  birth 
year  with  Y  but  then  X  and  Y  shared  their  information. 
A  simple  approach  to  preventing  this  would  be  to  model 
adversary  knowledge  globally,  effectively  assuming  that  all 
queriers  share  their  query  results;  doing  so  would  prevent  ei¬ 
ther  X’s  or  Y’s  query  (whichever  was  last).  This  approach  is 
akin  to  having  a  global  privacy  budget  in  differential  privacy 
and,  as  there,  obviously  harms  utility.  Dealing  with  collusion 
is  more  problematic  when  using  probabilistic  queries,  e.g.. 
Example  2.  This  is  because  highly  improbable  results  make  a 
querier  more  uncertain,  so  combining  querier  knowledge  can 
misrepresent  individual  queriers’  beliefs.  Roughly  speaking, 
querier  X  could  perform  a  query  Q  that  misinforms  the 
modeled  global  belief,  but  since  querier  F’s  actual  belief  is 
not  changed  by  the  result  of  Q  (since  he  did  not  actually  see 
its  result),  he  could  submit  Q'  and  learn  more  than  allowed 
by  the  threshold.  Disallowing  probabilistic  queries  solves 
this  problem  but  harms  expressiveness.  Another  option  is  to 
more  actively  track  a  set  of  beliefs,  as  hinted  at  above. 


4By  our  calculations,  with  privacy  parameter  e  =  0.1  recommended 
by  Dwork  [24],  the  probability  the  query  returns  the  correct  result  is 
approximately  0.5249. 


IX.  Conclusion 

This  paper  has  explored  the  idea  of  knowledge-based 
security  policies:  given  a  query  over  some  secret  data,  that 
query  should  only  be  answered  if  doing  so  will  not  increase 
the  querier’s  knowledge  above  a  fixed  threshold.  We  enforce 
knowledge-based  policies  by  explicitly  tracking  a  model 
of  a  querier’s  belief  about  secret  data,  represented  as  a 
probability  distribution,  and  we  deny  any  query  that  could 
increase  knowledge  above  the  threshold.  Our  denial  criterion 
is  independent  of  the  actual  secret,  so  denial  does  not 
leak  information.  We  implement  query  analysis  and  belief 
tracking  via  abstract  interpretation  using  novel  domains 
of  probabilistic  polyhedra  and  powersets  of  probabilistic 
polyhedra.  Compared  to  typical  approaches  to  implementing 
belief  revision,  our  implementation  using  this  domain  is 
more  efficient  and  scales  better. 
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Appendix  A. 

Concrete  probabilistic  semantics 

Here  we  briefly  explain  the  concrete  probabilistic  seman¬ 
tics  given  in  Figure  3.  More  details  can  be  found  in  Clarkson 
et  al.  [9]. 

The  semantics  of  skip  is  straightforward:  it  is  the  identity 
on  distributions.  The  semantics  of  sequences  Si  ;  S2  is  also 
straightforward:  the  distribution  that  results  from  executing 
S±  with  5  is  given  as  input  to  S2  to  produce  the  result. 

The  semantics  of  assignment  is  <5  [at  — >2?],  which  is 
defined  as  follows: 

6  [x  — »  E\  =f  Act.  S(t) 

t  I  t[x->[B]t]=<t 

In  words,  the  result  of  substituting  an  expression  E  for  x 
is  a  distribution  where  state  a  is  given  a  probability  that  is 
the  sum  of  the  probabilities  of  all  states  r  that  are  equal 
to  cr  when  x  is  mapped  to  the  distribution  on  E  in  r. 
For  implementation  purposes,  it  will  be  useful  to  consider 
separately  the  case  where  assignment  is  invertible. 

When  x  — ^  E  is  an  invertible  transformation,  the  formula 
for  assignment  can  be  simplified  to  the  following,  where 
x  — +  E'  is  the  inverse  of  x  — »  E. 

S[x  —y  E]  =  Act.  <5(ct  [x  -►  [£>]) 

When  x  — »  E  is  not  invertible,  the  original  definition  is 
equivalent  to  a  projection  followed  by  an  assignment.  Let 
V'  =  domain{5)  —  {x]  and  let  S'  =  S  f  V' .  Then  we  have 
the  following  for  a  non-invertible  assignment. 

S  [x  — >  E]  =  Act.  if  a(x)  =  \E\a  then  S' (a  [  V')  else  0 

In  the  appendix,  we  show  that  this  definition  by  cases  is 
equivalent  to  the  original  definition  (Theorem  56). 
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The  semantics  for  conditionals  makes  use  of  two  operators 
on  distributions  which  we  now  define.  First,  given  distribu¬ 
tions  and  82  we  define  the  distribution  sum  as  follows: 

<5i  +  62  =  Act.  (5i(cr)  +  62(0') 

In  words,  the  probability  mass  for  a  given  state  a  of  the 
summed  distribution  is  just  the  sum  of  the  masses  from  the 
input  distributions  for  a.  Second,  given  a  distribution  8  and  a 
boolean  expression  B,  we  define  the  distribution  conditioned 
on  B  to  be 

8\B  =  Act.  if  \B\a  then  8(a)  else  0 

In  short,  the  resulting  distribution  retains  only  the  probability 
mass  from  8  for  states  cr  in  which  B  holds. 

With  these  two  operators,  the  semantics  of  conditionals 
can  be  stated  simply:  the  resulting  distribution  is  the  sum  of 
the  distributions  of  the  two  branches,  where  the  first  branch’s 
distribution  is  conditioned  on  B  being  true,  while  the  second 
branch’s  distribution  is  conditioned  on  B  being  false. 

The  semantics  for  probabilistic  conditionals  like  that  of 
conditionals  but  makes  use  of  distribution  scaling ,  which  is 
defined  as  follows:  given  8  and  some  scalar  p  in  [0,1],  we 
have 

p  ■  8  =  A  a.  p  ■  8(a) 

In  short,  the  probability  ascribed  to  each  state  is  just  the 
probability  ascribed  to  that  state  by  8  but  multiplied  by  p. 
For  probabilistic  conditionals,  we  sum  the  distributions  of 
the  two  branches,  scaling  them  according  to  the  odds  q  and 
1-q. 

The  semantics  of  a  single  iteration  of  a  while  loop  is 
essentially  that  of  if  B  then  S  else  skip  and  the  semantics  of 
the  entire  loop  is  the  fixed  point  of  a  function  that  composes 
the  distributions  produced  by  each  iteration.  That  such  a 
fixed  point  exists  is  proved  by  Clarkson  et  al.  [9], 

Finally,  the  semantics  of  uniform  x  ri\  ri2,  introduced  in 
Section  V  is  given  as 

[uniform  x  rii  ri2 ]<5  =  (8  \  V  —  {x})  x  8' 

Where  V  is  the  set  of  variables  of  8,  and  S'  is  defined  as 
follows. 

S'  =  A  a.  if  m  <  a(x)  <  712  then  - - -  else  0 

n2-n1  +  l 

Appendix  B. 

Alternative  (flawed)  threshold  security  policy 
As  an  alternative  to  Definition  3,  suppose  we  used  the 
following  instead: 

Vox  e  support(8'  \  L).  (normal^' |ctl  \  H))(oh)  <  t 

Here  is  an  example  that  illustrates  why  this  definition  is  not 
safe,  as  it  could  underestimate  the  information  a  querier  can 
learn. 


Suppose  Bob’s  threshold  for  his  birth  year  byear  is 
t  =  0.05.  He  models  a  social  networking  site  X  as  believing 
his  age  is  more  likely  between  20  and  40  than  between  40 
and  60,  e.g.,  1971  <  byear  <  1991  with  probability  0.6 
(thus,  0.03  per  possibility)  and  1951  <  byear  <  1971  with 
probability  0.4  (thus,  0.02  per  possibility).  If  user  Bob  was 
born  in  1965,  then  X’s  believes  his  is  actual  birth  year  not 
as  likely  a  more  recent  year,  say  1975;  in  any  case  X  does 
not  currently  believe  any  possibility  above  Bob’s  threshold. 
Now  suppose  X  submits  program  S  that  determines  whether 
Bob’s  birth  year  is  even.  The  revised  belief  will  include  only 
even  (when  output  =  True)  or  odd  (when  output  =  False) 
birthdays,  increasing  the  likelihood  of  years  in  the  range 
[1971, 1991)  to  be  0.06  per  point,  and  the  likelihood  of 
years  in  the  range  [1951,1971)  to  be  0.04  per  point.  Bob’s 
birthday  is  1965,  and  its  probability  0.04  is  less  than  t,  so 
according  to  the  flawed  definition  the  agent  would  respond  to 
this  query.  But  if  this  query  result  is  returned,  X  will  see  that 
there  are  ten  possibilities  of  birth  year  that  are  above  Bob’s 
threshold.  X  can  deduce  that  none  of  these  possibilities  is 
Bob’s  actual  birth  year,  or  else  the  query  would  have  been 
rejected.  Excluding  these  possibilities,  he  knows  that  Bob’s 
birth  year  is  one  of  ten  possibilities  between  1951  and  1971 
ascribing  to  each  a  probability  0.1  which  exceeds  Bob’s 
threshold  of  0.05. 

Appendix  C. 

Example  queries 

We  provide  here  the  queries  and  prebeliefs  we  used  for 
the  experiments  in  Section  VII.  The  queries  are  described 
as  functions  from  some  set  of  inputs  to  some  set  of  outputs. 
The  exact  syntax  is  as  follows. 

querydef  queryname  ini  •  ■  •  inn  — >  out\  ■  ■  ■  outm  : 
querybody 

To  specify  a  query  invocation  we  use  the  following  syntax. 

query  queryname  : 
in\  valp, 

XTtfi  . —  Valn 

Each  experiment  must  also  specify  the  values  of  the 
secrets  being  queried,  and  the  querier’s  prebelief.  Each  spec¬ 
ification  is  a  merely  a  program  that  sets  the  values  of  these 
variables.  For  the  actual  secret  values  this  program  begins 
with  the  declaration  secret;  the  resulting  state  of  executing 
program  is  taken  to  be  the  secret  state.  The  program  to  set 
the  prebelief  begins  belief  and  has  the  same  format;  note 
that  this  program  will  use  pif  or  uniform  x  ni  n2  to  give 
secrets  different  possible  values  with  different  probabilities. 

We  now  give  the  content  of  the  queries  used  in  the 
experiments. 
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1 )  Birthday:  For  the  small  stateset  size  birthday  experi¬ 
ments  we  used  the  following  secret  and  prebelief. 

secret  : 

s_hday  :  =  270  ; 
sjbyear  :=  1980 

belief  : 

uniform  s_bday  0  364  ; 
uniform  sjbyear  1956  1992 

The  two  queries  used  were  as  follows. 

querydef  bday  :  c_day  — >  output 

if  sjbday  >  c_day  A  c_day  +  7  >  s_bday  then 
output  :=  1 

else 

output  :=  0 

querydef  spec  :  cjyear  — »  output 
age  :=  cjyear  —  sjbyear  ; 

if  age  =  10  V  age  =  20  V  age  =  30  V  age  =  40  V  age  =  50  then 
output _temp  :=  1 
else 

output _temp  :=  0  ; 

pif  1/10  then 

output  :=  1 

else 

output  :=  output _temp 

The  statistics  shown  include  the  time  spent  processing  this 
initial  setup  as  well  as  the  following  sequences  of  queries. 

•  A  single  bday  query  alone. 

query  bday  : 
c_day  :=  260 

•  Two  bday  queries. 

query  bday  : 
cjday  :=  261 

•  Two  bday  queries  followed  by  a  spec  query. 

query  spec  : 
cjyear  :=  2011 

2)  Birthday  (large):  For  the  larger  statespace  birthday  ex¬ 
ample  we  used  the  following  secret  and  prebelief  generators. 

secret  : 

s_bday  :=  270  ; 
sjbyear  :=  1980 

belief  : 

uniform  sjtday  0  364  ; 
uniform  sjtyear  1910  2010 

The  queries  used  were  identical  to  the  ones  for  the  smaller 
statespace  birthday  example. 


3)  Pizza:  The  pizza  example  is  slightly  more  compli¬ 
cated,  especially  in  the  construction  of  the  prebelief.  This  ex¬ 
ample  models  a  targeted  Facebook  advertisement  for  a  local 
pizza  shop.  There  are  four  relevant  secret  values.  The  level 
of  school  currently  being  attended  by  the  Facebook  user  is 
given  by  s_in_school_type,  which  is  an  integer  ranging 
from  0  (not  in  school)  to  6  (Ph.D.  program).  Birth  year 
is  as  before  and  s_address_lat  and  s_address_long 
give  the  latitude  and  longitude  of  the  user’s  home  address 
(represented  as  decimal  degrees  scaled  by  a  factor  of  106 
and  converted  to  an  integer). 

The  initial  belief  models  the  fact  that  each  subsequent 
level  of  education  is  less  likely  and  also  captures  the 
correlation  between  current  educational  level  and  age.  For 
example,  a  user  is  given  an  approximately  0.05  chance  of 
currently  being  an  undergraduate  in  college,  and  college 
attendees  are  assumed  to  be  born  no  later  than  1985  (whereas 
elementary  school  students  may  be  born  as  late  as  2002). 

rendering  latex 

secret  : 

s_in_school_type  :=  4  ; 
sjoirthjyear  :=  1983  ; 
s_addressjat  :=  39003178  ; 
s_addressJong  :=  —76958199 

belief  : 

pif  4/24  then 

uniform  s_in_school_type  1  1  ; 
uniform  sjoirthjyear  1998  2002 
else 

pif  3/19  then 

uniform  s_in_school_type  2  2  ; 
uniform  sjoirthjyear  1990  1998 
else 

pif  2/15  then 

uniform  s_in_school_type  3  3  ; 
uniform  s_birth_year  1985  1992 
else 

pif  1/12  then 

uniform  s_in_school_type  4  4  ; 
uniform  sjoirthjyear  1980  1985 
else 

uniform  s_in_school_type  0  0  ; 
uniform  sjoirthjyear  1900  1985  ; 
uniform  s_addressjat  38867884  39103178  ; 
uniform  s_address_long  —77058199  —  76825926 

The  query  itself  targets  the  pizza  advertisement  at  users 
who  are  either  1)  in  college,  2)  aged  18  to  28,  or  3)  close  to 
the  pizza  shop  (within  a  square  region  that  is  2.5  miles  on 
each  side  and  centered  on  the  pizza  shop).  If  any  of  these 
conditions  are  satisfied,  then  the  query  returns  1,  indicating 
that  the  ad  can  be  displayed.  The  full  text  of  the  query  is 
given  below. 

querydef  pizza  :  — >  output 
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if  s_in_school_type  >  4  then 
in_school  :=  1 

else 

in_school  :=  0  ; 
age  :=  2010  —  s_birthjyear  ; 
if  age  >  18  A  age  <  28  then 
age_criteria  :=  1 

else 

age_criteria  :=  0  ; 
lr_lat  :=  38967884  ; 
uljat  :=  39003178  ; 

Irjong  :=  -76958199  ; 
ul_long  :=  —76925926  ; 
if  s_address_lat  <  ul_lat  A 
s_address_lat  >  lr_lat  A 
s_address_long  >  lr_long  A 
s_address_long  <  ul_long  then 
injbox  :=  1 
else 

injbox  :=  0  ; 

if  ( in_school  =  1  V  age_criteria  =  1)  A 
injbox  =  1  then 
output  :=  1 

else 

output  :=  0 

4)  Photo:  The  photo  query  is  a  direct  encoding  of 
a  case  study  that  Facebook  includes  on  their  advertising 
information  page  [30].  The  advertisement  was  for  CM 
Photographies,  and  targets  offers  for  wedding  photography 
packages  at  women  between  the  ages  of  24  and  30  who 
list  in  their  profiles  that  they  are  engaged.  The  secret  state 
consists  of  birth  year,  as  before,  gender  (0  indicates  male,  1 
indicates  female),  and  “relationship  status,”  which  can  take 
on  a  value  from  0  to  9.  Each  of  these  relationship  status 
values  indicates  one  of  the  status  choices  permitted  by  the 
Facebook  software.  The  example  below  involves  only  four 
of  these  values,  which  are  given  below. 

0  No  answer 

1  Single 

2  In  a  relationship 

3  Engaged 

The  secret  state  and  prebelief  are  as  follows. 

secret  : 

sjbirthjyear  :=  1983  ; 

s_gender  :=  0  ; 

sjr  elation  ship _status  :=  0 

belief  : 

uniform  sjbirthjyear  1900  2010  ; 

uniform  s _gender  0  1  ; 

uniform  sjrelationship_status  0  3 


The  query  itself  is  the  following. 

querydef  cm_advert  :  — ►  output 

age  :=  2010  —  s_birthjyear  ; 
if  age  >  24  A  age  <  30  then 
age_sat  :=  1 
else 

age_sat  :=  0  ; 
if  s _gender  =  1  A 

sjr  elation  ship _status  =  3  A 
age_sat  =  1  then 
output  :=  1 
else 

output  :=  0 

5)  Travel:  This  example  is  another  Facebook  advertising 
case  study  [31].  It  is  based  on  an  ad  campaign  run  by 
Britain’s  national  tourism  agency,  VisitBritain.  The  cam¬ 
paign  targeted  English-speaking  Facebook  users  currently 
residing  in  countries  with  strong  ties  to  the  United  Kingdom. 
They  further  filtered  by  showing  the  advertisement  only  to 
college  graduates  who  were  at  least  21  years  of  age. 

We  modeled  this  using  four  secret  values:  country,  birth 
year,  highest  completed  education  level,  and  primary  lan¬ 
guage.  As  with  other  categorical  data,  we  represent  language 
and  country  using  an  enumeration.  We  ranked  countries  by 
number  of  Facebook  users  as  reported  by  socialbakers.com. 
This  resulted  in  the  US  being  country  number  1  and  the 
UK  being  country  3.  To  populate  the  list  of  countries  with 
“strong  connections”  to  the  UK,  we  took  a  list  of  former 
British  colonies.  For  the  language  attribute,  we  consider  a 
50-element  enumeration  where  0  indicates  “no  answer”  and 
1  indicates  “English”  (other  values  appear  in  the  prebelief 
but  are  not  used  in  the  query). 

secret  : 

country  :=  1  ; 
birthjyear  :=  1983  ; 
completed_school_type  :=  4  ; 
language  :=  5 

belief  : 

uniform  country  1  200  ; 
uniform  birthjyear  1900  2011  ; 
uniform  language  1  50  ; 
uniform  completed_school_type  0  5 

querydef  travel  :  — >  output 
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if  country  =  1  V  country  =  3  V 
country  =  8  V  country  =  10  V 
country  =  18  then 
main_country  :=  1 
else 

main_country  :=  0  ; 
if  country  =  169  V  country  =  197  V 

country  =  194  V  country  =  170  V 

country  =  206  V  country  =  183  V 

country  =  188  then 

island  :=  1 
else 

island  :=  0  ; 

age  :=  2010  —  birthjyear  ; 
if  language  =  1  A 

(■ main_country  =  1  V  island  =  1)  A 
age  >  21  A 

com.pleted_school_type  >  4  then 
output  :=  1 
else 

output  :=  0 

Appendix  D. 

Soundness  proofs  for  P 

A.  Projection 

The  proof  of  projection  relies  heavily  on  splitting  up  the 
support  of  a  distribution  into  equivalence  classes  based  on 
the  states  they  project  to.  We  will  have  a,  o'  G  supported) 
belonging  to  the  same  equivalence  class  iff  a  \  V  =  a'  \  V. 
The  details  are  formalized  in  the  following  definition. 


Definition  37.  Equivalence  classes  under  projection. 

•  [°v]]f  is  an  equivalence  class  of  elements  of  support(S) 
that  project  to  cry  (when  projected  to  variables  V). 

Formally,  [ay]Y  =  {cr  G  supported)  \  a  f  V  =  cry}. 

- v 

•  lav}s  's  a  subset  of  supported)  that  project 

to  anything  but  cry  or  formally  [ay]5  == 

{cr  G  support^)  \  <j  \  V  cry}. 

•  Me  *s  a  subset  of  7c(C)  that  project  to  cry 
(when  projected  to  variables  V").  Formally,  IcrylY  =f 
{cr  Gp't(C)  |  cr  \  V  =  cry}. 

•  [av]c  is  a  subset  of  7c (C)  that  project  to  anything  but 

- V  def 

cry  or  formally  [oy)c  =  {a  G  7c(C)  |  cr  [  V  f  cry}. 

Remark  38.  Let  V  C  V'  C  fv(S),  a  G  support(S).  and 
cry,  <j'v  G  supported  \  V). 

(i)  (a  r  V')  \  V  =  a  r  V 

The  sets  of  t  [ayjT  >  form  a  partition  of 

^  '  ay  (z  support  (5 \V) 

support(S),  equivalently  the  following  two  claims. 

(ii)  support {5)  =  [}avesupportWV)  Mil 


(iii)  [cry]  J  n  [<7y]^  =  0  whenever  cry  a'v 
Likewise,  for  any  cry  G  supported  \  V),  the  sets  of 

1  ky'lj”  c  form  a  partition  of  [cry]  J,  implying 

t  J  ay/Cr_  1 


>s\v' 

also  the  following  claim. 

(iv)  [av]J  =\Jav,e[<7V]v 


,  [ay 


iV' 


The  equivalence  classes  in  terms  of  the  concrete  support 
sets  as  related  to  the  abstract  support  sets  are  expressed  in 
the  following  manner. 

(v)  [ay]s  Q  [ay]c  an^  lav}s  —  [ay]c  whenever 
supported)  C  7c(C') 

Finally,  the  concrete  projection  operation  can  be  rewritten 
in  terms  of  the  equivalence  classes,  a  fact  we  will  repeatedly 
use  in  the  proofs  to  follow  without  explicitly  stating  it. 

(vi)  (5  r  v  =  Aery.  6(a) 

Proof:  All  of  these  are  merely  expansions  of  the  various 
definitions  involved.  Note  that  the  two  parts  of  Remark 
38  (v)  are  not  contradictory  as  [ay]5  and  [ crv]s  are  not 
set  complements  of  each  other  when  viewed  as  subsets 
of  7c  (C),  though  they  are  complements  when  viewed  as 
subsets  of  support(S).  ■ 

Lemma  39  (Conservation  of  Mass).  If  V  C  fv(S)  then 

n*n  =  ii*m. 

Proof:  Let  us  consider  the  terms  of  the  projected  mass 

sum. 

11*  t  v\\  = 

ay  0zsupport(5  \V) 

=  *(*) 
cr(zSupport(5) 

=  11*11 


The  terms  in  the  double  sum  are  the  same  as  those  in  the 
single  sum  as  all  terms  of  the  first  are  accounted  for  in  the 
second  due  to  Remark  38  (ii)  and  none  are  double  counted 
due  to  Remark  38  (iii).  ■ 


Definition  40.  Concrete  forget  can  be  defined  in  terms  of 
a  projection  to  all  but  one  variable.  That  is,  fx(5)  =  6  \ 
MS)  -  {a-}.  Also,  fXl)...  ,Xn(6)  =  fX2,...,Xn(f Xl(S)). 

The  correspondence  between  repeated  concrete  forget  and 
a  projection  involving  removal  of  more  than  one  variable  will 
be  demonstrated  shortly. 

Lemma  41  (Order  of  Projection).  If  V  C  V'  C  fv(S)  then 
(6  [  V’)  \  V  =  6  {  V  . 

Proof:  Let  cry  G  <5  [  V . 


((*  r  v)  r  v)  (ay) 


£ 

°V'S[<T  v]gfVi 


£  s(°) 


V 


£  s(v) 

o-e[avlY 
(a  [  V)  (ay) 


(1) 

(2) 

(3) 

(4) 
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Let  P2  =  iy(P)-  P2  thus  has  the  following  properties. 


The  collapse  of  the  double  sums  on  (1)  to  (2)  is  due  to 
the  correspondence  between  the  terms  of  the  double  sum 
and  the  single  sum  due  to  Remark  38  (ii)  and  Remark  38 
(iii).  The  equality  of  the  union  of  equivalence  classes,  (2)  to 
(3)  is  due  to  Remark  38  (iv).  ■ 

Corollary  42.  <5  |  V  =  fXu—  ,Xtl(<5)  where  fv{8)  —  V  = 

{ah,  •  •  •  ,xn}. 

Proof:  Let  us  show  this  by  induction  on  the  size  of 
V  =  fv{5)  —  V.  When  |  V|  =  0  or  |V|  =  1,  the  claim  holds 
vacuously  or  by  definition  of  concrete  forget,  respectively. 

Let  us  assume  the  claim  for  V  |  =  to— 1  <  n  and  consider 
the  case  when  I V I  =  to  <  n. 


C2  =  f y(C) 

pmm  =  pmin  .  max  {hmin  _  (#(C)  _  s-in)  ^ 
min  _  r  min  /ymaxi 

b2  —  |o  /n2/  I 
s^lax  =  min{#(C'2),smax} 
m™in  =  rnmin 
m“ax  =  rnmax 


p™ax  =  pmax  .  lnin  {h™ax,  s 


(9) 

(10) 

(11) 

(12) 

(13) 

(14) 

(15) 


The  quantities  h“ln  and  h™ax  are  defined  to  exhibit  the 
following  properties. 


S  \  V  =  (8  \  V  U  {aq})  \  V  [by  Lemma  41  ] 

=  tXl{8)  fC 

=  f®2,-  ,Xm  (f*i  (<5))  [  by  induction  ] 

=  fxu- ,xm{8) 

Thus,  by  induction,  the  claim  holds  for  m  =  n.  ■ 

Remark  43  (Counting  Variations).  Two  simple  counting 
arguments  are  required  for  the  further  proofs. 

(i)  If  to  objects  are  distributed  fully  into  two  bins,  with  one 
of  the  bins  having  space  for  no  more  than  a  objects, 
then  the  other  must  have  at  least  m  —  a  objects  in  it. 

(ii)  If  m  objects  are  to  be  packed  into  bins  of  sizes 
Oi,  ■  ■  •  ,  an,  with  di  >  to,  the  least  number  of  bins 
that  can  be  used  to  fit  all  m  objects  is  greater  or  equal 
to  \m/a*]  where  a*  >  max,;  a,. 

Proof:  Part  (i)  is  immediate.  For  part  (ii),  consider  some 
optimal  set  of  bins  used  to  pack  the  m  objects.  This  set  of 
bins  would  also  let  one  pack  to  items  assuming  each  bin  had 
space  for  exactly  a*  objects  as  this  is  an  upper  bound  on  the 
size  of  each  bin.  Thus  the  space  of  solutions  to  the  original 
packing  problem  is  a  subset  of  the  space  of  solutions  to  the 
altered  packing  problem  where  all  bins  are  increased  to  fit 
a*  items.  Thus  the  solution  for  the  original  cannot  use  fewer 
bins  than  the  optimal  solution  for  the  altered  problem.  For 
this  alternate  problem,  the  minimum  number  of  bins  used  to 
pack  all  to  items  is  exactly  |~m/a*~|  by  a  generalization  of 
the  pigeonhole  principle.  ■ 

Lemma  7  (Soundness  of  Forget).  If  5  G  Jp(P)  then  fy((5)  € 

7r(f y(P))- 

Proof:  Let  8  £  7p(-P),  V  =  fv{8)  —  {y},  and  S2  =  8  f 
V.  By  assumption  8  has  the  following  properties. 


support(8 )  C  7 c(C)  (5) 

srnin  <  \support{8)\  <  smax  (6) 

mmin  <  ||<?||  <  rnmax  (7) 

Vo-  G  support {8)  .  pmin  <  <5(a)  <  pmax  (8) 


h™  <  min  [av]vc  (16) 

<7  v  G'Yc  (^2 ) 

h“ax  >  max  [crv]c  (17) 

<7vS7c(G2) 

To  show  that  82  G  7p(f y(P))  we  need  to  show  the 
following. 


support(S2)  C  70(672)  (18) 

s“in  <  \support(S2)\  <  srx  (19) 

m™in  <  ||(52||  <  m™ax  (20) 

Vay  G  support{82)  ■  p™in  <  <52(<Jy)  <  p™ax  (21) 
Let  us  show  each  of  these  in  turn. 


Claim  (18)  -  Support.  Let  ay  G  support{82).  Thus 

82(<Jv)  =  Eaekv- ]Y  >  0  so  there  exists  a  e  iav] % 
with  (5(a)  >  0.  So  a  G  supported).  Therefore,  by  (5), 
a  G  7c(67),  therefore  ay  G  7c(672)  by  definition  of 
polyhedron  forget.  Therefore  support(S2)  C  7c(672).  □ 

Claim  (19)  Support  points.  First  let  us  show  the  follow¬ 
ing  claim. 

max  [ay][  <  h“ax  (22) 

cry  £support(52) 

By  construction  of  h™ax,  we  have  h“ax  > 
max0.ve7c(C-2)  [ay]^  .  Now,  support(82 )  C  jc(C2) 
by  (18).  Also  for  any  ay  G  support(S2),  we  have 
[ay]J  C  [ffyjp  by  Remark  38  (v).  Therefore 

maX(T y  G7C  (C2  )  [oy]c  V  maxCTV  (^support(J)2 )  Wv]  y  — 
maxCTl/ esupport(s)  [ay][  .  Thus  concluding  h“ax  > 

maxCT^  ^support  (<52 )  ^  • 

Consider  the  elements  of  supported)  as  they  map  via 
state  projection  to  elements  of  support{82 )•  Let  us  view 
the  elements  of  the  later  as  bins,  with  the  elements  of 
the  former  as  objects  to  pack  into  the  bins.  By  (22), 
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we  know  no  bin  has  more  than  h“ax  objects,  thus  we 
can  apply  Remark  43  to  conclude  there  are  at  least 
\\support(8)\  /li'jlax]  non-empty  bins,  or  in  other  words, 
\support(S2)\  >  \\support{8)\ /h™ax].  This  is  itself  at 
least  as  large  as  |~smm/h“ax]  =  s“ln  by  (6).  Therefore 
\support(S2)\  >  s™1". 

For  the  other  side  of  the  inequality,  note  that  the  number 
of  bins  used,  or  Isupportff-f) |  cannot  exceed  \support(8)\  < 
smax  itself.  It  also  cannot  exceed  |7c(C<2)|  =  #(6/2) 
given  (18).  Therefore  support (62)  <  min  {#((72), smax}, 
concluding  requirement  (19).  □ 

Claim  (20)  Mass.  This  requirement  holds  trivially  due  to 
Lemma  39  and  assumption  (7).  □ 


Claim  (21)  Probability.  Let  us  first  show  the  following 
claim. 


min 

ay  £support(S 2  ) 


>  hmm 


+  s 


min 


#(G> 
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Let  cry  G  support (62).  Let  us  consider  the  size  of  [cry}5  . 


- v 

Ms 


< 


Me 


=  #(C)  -  [Me 


<  #(C)  —  min 

Tv€l'1c(C2) 

<#(C)-  h“in 


[  by  Remark  38  (v)  ] 


[  by  (16)  ] 


Let  us  view  now  the  elements  of  supported)  as  map¬ 
ping  (via  projection)  into  two  bins,  [ov]^  and  [cry]s  . 
By  the  argument  above,  we  know  the  second  bin  cannot 
hold  more  than  #(C)  —  h“ln  elements,  thus,  by  Remark 
43  (i),  it  must  be  the  case  that  the  first  bin  contains  at 
least  \support{5)\  —  (#(C)  —  h™ln)  elements.  This  itself 
is  no  smaller  than  smm  —  #(C)  +  h™m  by  (6).  Therefore 

rv]V  >  smin  -  #(C)  +  li"lin  and  thus  claim  (23)  holds. 
Consider  now  cry  £  support(S2 )■  By  (8)  and  the  concrete 
projection  definition,  it  must  be  the  case  that  62  (cry)  = 

Eaekvir  M)  -  Pmin-  Also’ 


S2{crv)  = 

E 

(5(cr) 

t 

re[<rv]j' 

> 

E. 

pmin 

[  by  (8)  ] 

t 

re[<rv]y 

= 

Mir 

•  pmin 

>  1 

(lr“in  +  smin  - 

-  #(C))  •  Pmin  [  by  (23)  ] 

Therefore,  82 (cry) 

>  pmin 

•min  {1,  h“in  +  smin  -  #(C) } 

=  p™ln,  concluding  one  inequality  of  the  last  condition. 

For  the  other  inequality,  let  us  once  more  consider  a 
general  ay  €  support (<j2). 


MM  =  6 (CT) 

cr€[<TV]X 

<  y  pmax 

<re[(rv]s 


Mil 


•p 


<  h“ax  •  pmax 


t  by  (8)  ] 


[  by  (22)  ] 


Since  [a y]^  C  support(S),  we  have  [cry]y 


< 


^  vv  navu 

\support(8)\  <  smax  (by  (6)).  Thus  we  can  also  bound 
M°v)  by  smax  •  pmax.  Therefore,  $2  (cry)  <  pmax  . 

},  completing  the  last  claim.  □ 


min  {h;;lax,  smax 


Lemma  44  (Soundness  of  Projection).  If  6  £  'yp(P)  then 
6  r  V  G  7P  (P  \  V). 

Proof:  Let  us  show  this  by  induction  on  the  size  of 
V  ==  fv{8)  —  V.  When  |V|  =  0  there  is  no  projection  to 
be  done,  when  |V|  =  1,  the  claim  holds  by  Lemma  7.  Let 
us  assume  the  claim  holds  for  |  V  |  =  n  —  1  and  look  at  the 
case  where  |C|  =  n. 

Let  us  write  V  =  {xi,---  ,xn}.  Thus  S  f  V  = 
iXlt ...  ,Xn  (8)  by  Corollary  42.  By  definition  of  for¬ 
get,  we  also  have  f Xl,-,Xn(S)  =  fX2l...  lXn  (fXl  (5))  and 
f*i ,-,xn(-P)  =  fx2,-,x„(fxi(-P))-  By  Lemma  7,  we  know 
that  fXl(<5)  £  7p(fXl  (-P)),  therefore,  by  induction,  S  f  V  = 
fX2,...  ,Xn{fXl(8))  £  fX2l...,In(fXl(P))  =P\V.  U 

B.  Assignment 

We  begin  with  some  useful  notation. 

Notation  45.  Let  cr  be  a  state,  E  be  an  expression,  a;  be  a 
variable,  S  C  State,  V  C  Var. 

•  a  [x  — >  E]  =  0  [x  — *  \E\cr] 

.  S[x-+E\  =  {a  [x  ->  E\  |  a  £  S}. 

.  s  \  V  =  {cr  \  V  I  CT  G  S} 

Definition  46.  A  state  cr  is  feasible  for  x  —r  E  iff  cr  £ 
State  \x  — >  E}.  We  will  say  that  cr  is  merely  feasible  if  the 
assignment  is  clear  from  the  context. 

Definition  47.  t,:_  e  is  the  function  from  State  to  feasible 
states  (for  x  — >  E)  defined  by  ■  cr  1— >  0  [x  — »  E\ 

Definition  48.  The  inverted  equivalence  class  for  cr  under 
assignment  x  — »  E  is  the  set  of  states  that  map  to  cr.  We 
define  two  varieties,  one  over  all  possible  states  and  one  for 
just  the  states  in  the  support  of  a  distribution. 

.  (a)x^E  =  {r  |  r  [x  -*  E]  =  a} 

•  (a)yE  ==  {T  ^  support^)  |  r  [x  — >  E\  =  a} 

Note  that  cr  is  feasible  iff  (a)x^E  7^  0. 

Definition  49.  An  assignment  x  — >  E  is  invertible  iff  tx^E 
is  invertible.  We  will  denote  tffE  as  the  inverse  of  if 
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it  is  invertible.  The  invertability  of  tx—,E  is  characterized  by 
the  existence  of  the  inverse,  having  the  property,  that  for  ev¬ 
ery  a  G  State,  we  have  t~^E  (tx^E  (c))  =  cr.  Equivalently, 
for  every  feasible  state  cr,  tx^E  tei*  w)  =  cr. 

We  can  also  characterize  invertability  via  inverted  equiv¬ 
alence  classes,  x  — +  E  is  invertible  iff  for  every  feasible  cr, 

|(cr)a:^-E|  _ 

We  will  say  E  is  invertible  if  the  variable  is  clear  from 
the  context. 

Note  that  since  tx^E  only  changes  the  x  component  of  a 
state,  the  inverse,  t~_^E,  also  only  changes  the  x  component, 
if  the  inverse  exists.  This  doesn’t  mean,  however,  that  the 
inverse  can  be  represented  by  an  assignment  of  some  E' 
to  x.  Furthermore,  since  our  language  for  expressions  lacks 
division  and  non-integer  constants,  no  assignment’s  inverse 
can  be  represented  by  an  assignment. 

Definition  50.  The  expression  E  is  integer  linear  iff  E  = 
ni  x  x\  +  ■  ■  ■  T  nm  x  xm,  where  n*  are  integer  constants, 
and  Xi  are  variables.  We  assume  that  all  the  variables  in  a 
given  context  are  present.  We  will  generally  use  xt  and  n, 
to  refer  to  the  contents  of  a  integer  linear  expression. 

From  now  on,  we  will  assume  all  expressions  E  are 
integer  linear.  Programs  containing  non-linear  expressions 
are  just  not  handled  by  our  system  at  this  stage  and  linear 
expressions  not  fully  specified  are  equivalent  to  integer  linear 
expressions  with  n,  =  0  for  variables  unused  in  the  original 
expression. 

Lemma  51.  Xi  — ►  E  is  non-invertible  iff  ri  \  =  0.  In  other 
words,  X\  — >  E  is  non-invertible  iff  E  doesn’t  depend  on 
xi. 

Proof:  (=>)  Assume  otherwise.  Thus  E  is  non-invertible 
but  rii  0.  So  we  have  a  feasible  state  cr  with  |  {o)Xl^E\  ^ 
1.  Since  feasible  states  have  non  empty  inverted  equivalence 
sets,  it  must  be  that  |(cr)Xl^'B|  >  2.  So  let  r,  r'  G  (a)Xl^E 
with  t  t' .  So  r  [x\  — >  E]  =  t'  [x\  — >  E]  =  a.  Since 
assignment  to  X\  doesn’t  change  the  state  other  than  in  its 
value  of  ati,  r  and  r'  can  only  differ  in  their  value  for  x  \ . 

But  since  r  and  t'  are  identical  after  the  assignment,  we 
have, 

T  [xi  — *■  E\  (aii)  =  n1r{xi)  +  n2r( x2)  H - b  nmr( xm) 

=  Uit(x1)  +  n2T(x 2)  -I - b  nmr\xm ) 

=  n\T'{x\)  +  n2r'( x2)  H - b  nmr' (xm) 

=  t'  [xi  — *■  E\  (xi) 

Canceling  out  the  common  r'(xi)  terms,  we  have 
nir(xi)  =  niT'(xi )  and  since  n\  0,  we  conclude 
t(x\)  =  r'(x  1),  contradicting  r  t'. 

(4=)  Let  cr  be  a  feasible  state  and  let  r  G  (a)Xl^E. 
Let  t'  =  r  [x\  — >  t(x  1)  +  1].  Since  E  doesn’t  depend  on 
Xi,  we  have  [£I]t  =  \E\t’  and  therefore  r'  [x\  — *  E\  = 


t  [xi  — >  E]  =  o  and  so  we  have  r,  t'  G  (o)Xl^E  with 
r  /  t',  therefore  E  is  non-invertible.  ■ 

Lemma  52.  Assume  x  — >  E  is  non-invertible.  cr  is  feasible 
iff  °  [%  ->  E\  =  cr. 

Proof:  (=>)  Let  cr  be  feasible.  Thus  a  =  t[x  E\  for 
some  t  G  State.  Since  E  doesn’t  depend  on  x  by  Lemma 
51,  we  have  (r  [x  — *  E\)  [x  —>  E]  =  r  [x  — ►  E\  =  a.  So 
a[x  — ►  E]  =  a. 

(^=)  Assume  a  [x  — >  E]  =  cr.  Thus  cr  G  State  [x  — ►  E\  by 

definition.  ■ 

Lemma  53.  Assume  x  — r  E  is  non-invertible.  Let  5  be  a 
distribution  with  x  £fv(S)  and  let  V  =  Jv(c 5)  —  {x}.  If  cr  is 
feasible,  then  (a)^E  =  [a  \  . 

Proof:  Let  r  G  So  r[i-»£]  =  cr  and 

t  G  supported).  But  the  assignment  only  changes  x,  thus 
t  \  V  =  a  \  V,  therefore  r  G  [cr  f  V]^ .  Thus  (cr)g^E  C 

[*  r  K- 

Let  r  G  [cr  f  V"]^.  So  t  G  support (5)  and  r  [  V  = 
a  \  V.  Since  E  doesn’t  depend  on  x,  we  have  r  [x  -*£]  = 
cr  [x  —>  E\  =  a\  the  second  equality  follows  from  Lemma  52 
as  cr  is  feasible  by  assumption.  So  r  G  (cr)s^E ■  Therefore 

[*  r  K  <=  w)rE- 

■ 

Remark  54.  Assume  x  — >  E  is  invertible.  For  every  feasible 
cr,  we  have  (a)x^E  =  {t~^E(a)}. 

Proof:  Invertability  tells  us  that  ( o)x~*E  has  only  one 
element.  The  function  tf  \E,  given  the  feasible  cr,  produces 
an  element  of  (a)x^E,  as  (t~^E(o))  [x  — »  E]  —  0.  ■ 

Definition  55.  We  define  an  alternate  means  of  assignment, 
5(x  — y  E).  Let  V  =  ff(5)  —  {x}. 

•  If  x  — *  E  is  invertible,  then 

S(x  — »  E)  =  A  cr.  if  cr  is  feasible 

then  (5 

else  0 

•  If  x  -x  E  is  not  invertible,  then 

5{x  — >  E)  =  Act.  if  cr  is  feasible 

then  5  f  V  (cr  [  V) 

else  0 

Lemma  56.  For  any  S,  S  [x  — >  E\  =  S(x  — *  E). 

Proof:  Let  5'  =  8  [x  — >  E\  and  5"  =  5(x  — >  E). 

S'(a)  =  H  ,5(t) 

r  |  t[x—*E]—c t 

=  Y,  ^ 

r6(^)rE 

Case  1:  x  —>  E  is  invertible 
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If  <7  is  feasible,  (a)x^E  has  only  one  element,  a~l  = 
t~l+E(cr),  by  Remark  54.  So  S' (a)  =  S(a_1)  =  S"(a).  Note 
that  when  cr"1  is  not  in  support(S)  then  S' (a)  =  0  =  S"(cr). 

If  a  is  not  feasible  then  (a)x^E  =  0  so  S'(a)  =  0  = 
5”{a). 

Case  2:  x  — >  E  is  non-invertible 

If  <j  is  feasible,  then  by  Lemma  53  we  have  (a)^E  = 

MnT- 

^V)  =  E  6 (r) 

reW)rE 

=  5(r) 

T^W\v]f 

=  {5  r  V)  [a  r  V) 

=  5"  (a) 

If  a  is  not  feasible  then  (cr)$^E  =  0  so  S'(a)  =  0  = 
6"  (a). 

■ 

Lemma  57.  Assume  x  —>  E  is  invertible,  then  supported)  = 
cr)  |  cr  G  support (5 (x  — >  P))}. 

Proof:  Let  62  =  S(x  — >  E).  Let  r  G  support(S).  So 
a  =  t[x  —r  E\  €  support (<52)  and  f“^E(cr)  =  r.  So  r  G 
{dE(o-)  |  cr  G  support^)}. 

Let  r  G  |f“^E(cr)  |  cr  G  support(S2) }.  So  r  =  (cr) 
for  some  cr  G  supported 2).  So  there  exists  r'  G 
support(S)  such  that  r'  [x  — >  E]  =  a.  But  t~^E(a)  = 
tf\E{tx^E{r’))  =  t'  so  t'  =  r  as  r  =  f“iB(cr).  So 
r  G  support (6).  ■ 

Lemma  9  (Soundness  of  Assignment).  If  6  £  'yp(P)  then 
S  [. x  — *■  E\  G  7p  (P[x  — ►  P]). 

Proof:  Let  C  =  /v(<5)  —  {x}.  By  assumption,  we  have 
the  following. 

support(S)  C  7c  (C)  (24) 

smin  <  \support(S)\  <smax  (25) 

mmin  <  1 1 6 1 1  <  mmax  (26) 

Vcr  G  support {6)  .  pmin  <  6(a)  <  pmax  (27) 

Let  P2  =  P[x  — >  P]  and  <52  =  (5  [x  — »  P]  =  S(x  — >  P). 
Lemma  56  lets  us  use  <5  [x  — >  P]  or  S(x  — >  P)  interchange¬ 
ably. 

We  consider  two  cases.  Case  1:  x  — >  E  is  invertible 
In  this  case,  P2  is  defined  with  C2  =  C  [x  — »  P]  and 
all  other  parameters  as  in  P.  Thus  we  need  to  show  the 
following. 

support(S2 )  C  7c(C2)  (28) 

smin  =  s“in  <  \support(S2)\  <  srx  =  smax  (29) 

mmin  =  m“in  <  ||  ;2 1|  <  m“ax  =  mmax  (30) 

Vcr  G  support(S2)  .  pmin  =  p^in  <  62(a)  <  prx  =  Pmax 

(31) 


Claim  (28)  Support.  By  definition,  7c(C'2)  = 

{cr  [x  — ►  P]  |  cr  G  7c(C1)}.  Let  r  G  support(S2), 
so  we  have  cr  G  support(S)  C  7c  (C)  with 
cr  [x  — >  P]  =  r.  So  r  G  7c(C'2).  So  r  G  7c (C2)  and 
thus  support(S2)  C  7c(C2).  □ 

Claim  (29)  Support  points.  By  Lemma  57  we  have 
support(S)  =  {t~^E(a)  \  <7  G  i«pporf(52)}.  Inverse  func¬ 
tions  are  necessarily  injective  over  their  domain,  and 
since  support(S2)  are  all  feasible  (thus  in  the  domain  of 
the  inverse),  we  have  |  {i~_^B(cr)  |  cr  G  support^)}  |  = 
\support(S2)\.  So  \support(S)  =  \support(S2)\.  This,  to¬ 
gether  with  (25),  completes  the  claim.  □ 

Claim  (30)  Mass.  Note  again  that  support(S2)  C 
State  [x  — >  E\.  That  is,  all  possible  states  are  feasible.  So 
we  can  write: 

\M=  E  W 

cr£support(82 ) 

=  E  'KiEi?(cr))  [  by defn-  of  s2  ] 

a£support(52) 

=  E  r )  [  by  Lemma  57  ] 

r  ^support  (5) 

=  11*11 

The  above,  together  with  (26),  completes  this  claim.  □ 

Claim  (31)  Probability.  Since  support(S2)  are  feasible, 
we  have,  for  every  cr  G  support(S2),  S2(a)  =  S(t~^E  (o'))- 
But  also,  tx^E(a)  G  support(S).  Taking  this,  and  (27),  com¬ 
pletes  this  claim,  and  soundness  in  the  invertible  case.  □ 

Case  2:  x  — *  E  is  non-invertible  In  this  case,  P2 
is  defined  via  the  forget  operation.  If  Pi  =  f X(P)  and 
Ci  =  (Pi,  Ci),  then  P2  =  P[x  ->  P]  has  C2  =  (Pi  U 
{x  =  E}  ,  C  U  {x}),  and  all  other  parameters  as  in  Pi. 

We  need  to  show  the  following  four  claims. 

support(S2)  C  7c (C2)  (32) 

sfn  =  slT1  <  \support(S2)\  <  s^ax  =  s“ax  (33) 

111““  =  m“in  <  ||52 1|  <  m“ax  =  m“ax  (34) 

Va  G  support (62)  ■  pf"  =  <  S2(a)  <  p^ax  =  p?lax 

(35) 

Recall  the  definition  of  S2: 

S(x  — >  E)  =  Act.  if  cr  is  feasible 

then  5  \  V  (a  \  V) 

else  0 

Claim  (32)  Support.  Let  cr  G  siipport(S2).  So  cr  f  C  G 
supported  f  C).  so  there  exists  r  G  support(S)  C  7c(C) 
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with  t  \  V  =  a  \  V.  So  t  f  7  £  Tcft/C'))  =  7c(Ct)- 
So  t  £  7c((Pi>  Vi  U  {x}))  as  the  add  dimension  operation 
leaves  x  unconstrained.  The  non-constraint  of  x  also  tells  us 
that  a  £  7c((Pi,  Vi  U  {x}))  as  we  have  cr  f  V  =  r  \  V. 

Since  cr  g  support (62),  cr  is  feasible  so  cr  satisfies  the  x  = 
E  constraint  as  cr  =  r  [x  P]  for  some  r.  Thus,  overall, 
we  have  cr  g  7C((PiU{x  =  P}  ,  yiU{x}))  =  7c(C2).  □ 

Claim  (33)  Support  points.  Let  =  S  f  V  =  f:r(8).  By 

soundness  of  forget  (Lemma  7),  we  have  the  following. 

smi„  =  smin  <  \support^l)  \  <  S™X  =  S?““ 

All  we  need  to  show,  then,  is  the  following. 

\support(5i)\  =  \support(S2)\  (36) 

Let  us  show  this  by  establishing  a  bijection  /  between  the 
two  sets.  Let  us  define  /  :  supported  1)  — >  supported 2)  via 
/  :  av  ^  <jv  U  {x  =  \E\av}. 

To  show  /  is  injective,  let  cry,  cry  be  such  that  /(cry)  = 
/ ( cr y ) .  Since  /  does  not  change  any  part  of  the  state  other 
than  adding  x,  it  must  be  that  cry  =  a'v. 

To  show  that  /  is  suijective,  consider  cr  g  support (c>2). 
So  cr  is  feasible,  so  cr  [x  — >  P]  =  cr  by  Lemma  52.  Also 
cr  r  V  g  support(<Jv ),  considering  the  definition  of  cr2.  Since 
P  doesn’t  depend  on  x,  we  can  write  [P]cr  =  [P]cr  \  V , 
therefore  /(cr  f  V)  =  cr  f  C  U  {x  =  [P]cr  \ V}  = 
cr  [x  — >  P]  =  O. 

Since  /  is  injective  and  surjective,  it  is  a  bijection  and 
thus  \support{5\)\  =  \support(S2)  |.  □ 

Claim  (34)  Mass.  Let  <5i  =  S  \  V  =  fx(c)).  Let  us  show 
the  following  claim. 

LHS  =  support(Si)  =  {a  \  V  \  0  g  support (c>2)}  =  RHS 

(37) 

Let  cr y  g  supported  1).  So  there  exists  cr  g  support (<5) 
with  cr  f  V  =  cry.  So  cr  [x  P]  g  support (J2).  But  the 
assignment  doesn’t  change  anything  but  x,  so  it  must  be 
that  (cr  [x  — >  £7])  r  V  =  cr  f  V,  therefore  cry  =  cr  (  V  g 
{r  (  V  |  r  g  support(S2)}.  Thus  TPS'  C  RHS. 

On  the  other  side,  let  cr  g  support (<S2),  so  cr  =  r  [x  — >  E] 
for  some  r  g  supported),  by  the  original  definition  of 
distribution  assignment.  So  r  \  V  g  support(Si).  But 
(r[x— >P])  f  L  =  r  f  T  as  the  assignment  doesn’t 
change  anything  but  x.  So  cr  f  V  =  (r  [x  — >  P])  \  V  = 
t[x  — >  P]  g  support{5\),  concluding  that  RHS  C  LHS, 
and  thus  LfTS1  =  RHS. 

Note  that  this,  together  with  (36),  show  that  not  only  are 
the  sets  equal,  but  also  no  two  elements  of  support (<52) 
can  map,  via  projection  to  V,  to  the  same  element  of 
support(S[ ). 

By  soundness  of  forget  (Lemma  7),  we  have  the  following. 

m“in  =  m“in  <  1 1  <Si  ||  <  m“ax  =  m“ax 


Again,  we  proceed  to  show  that  || Aj ||  =  ||£2||. 

ini  =  E 

a  v  6  s  upport  ( 8 1 ) 

-  E  *MV0  [  by  (36)  and  (37)  ] 

l T(zSUpport(52 ) 

=  ^  <52  (cr )  [  by  defn.  of  82  ] 

a£support(52 ) 

=  INI 

□ 

Claim  (35)  Probability.  Let  cr  g  support(S2).  So  cr  is 
feasible,  so  82(u)  =  (S  \  V)  (cr  f  V)  >  0.  Therefore  cr  ( 
V  g  supported  \  V).  Thus,  by  soundness  of  forget  (Lemma 
7),  we  have  p“in  =  p“in  <  (<5  \  V)  (0  \  V)  <  p“ax  = 
p“ax,  concluding  the  claim  and  the  lemma.  □ 

■ 

C.  Plus 

Definition  58.  Let  overlap(Si,S2)  =  support(8\ )  fl 
support (82). 

Lemma  59.  If  Si  g  71?  (Pi)  and  82  £  7p(P2)  then  Pi  © 
P2  <  \overlap(8i,S2)\  <  Pi  ©  P2. 

Proof:  We  first  note  that  for  any  sets  A,  B,  it  is  the 
case  that  |  A  U  B\  =  |A|  +  \B\  —  \A  fl  B\  (often  called  the 
“inclusion-exclusion  principle”).  Rearranging  the  equation 
we  also  have  \A  fl  B\  =  \A\  +  \B\  —  \ A  U  B\. 

We  will  make  use  of  this  formula  with  A  =  support{8-\), 
B  =  support{82 )• 

Lower  Bound:  We  first  show  the  lower  bound.  Expand¬ 
ing  the  definitions  of  Pi  ©  P2  and  overlap{8\,  82),  this 
reduces  to  showing  the  following. 

max((s‘"in  —  ni)  +  (s“in  -  n2)  -  n3,  0) 

<  \support{8i)  fl  support{82)\ 

Clearly  we  have  0  <  \support(Si)  fl  supportlftf)],  so  it 
remains  to  show  that  the  following  holds. 

(sfn  -  ni)  +  (sfn  -  n2)  -  n3 

<  | support(Si)  C\  support(S2)\ 

Expanding  the  definitions  of  ni,n2  from  Definition  10, 
we  obtain 

(sfn  -  (#(Ci)  -  «3))  +  (srn  -  (#(C2)  -  n3))  -  n3 

<  \support(8i)  (8  support^)  \ 
and  rearranging  yields  the  following. 

s™in  +  s™in  —  {#{Ci)  +  #(C2)  —  n3) 

<  \support(8i)  fl  support (<52)  | 
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This  follows  from  the  rearranged  inclusion-exclusion 
principle  provided  we  can  show  s™ln  <  \support(Si)\, 
s“ln  <  \support(S2)\,  and  #(Ci)  +  #(C2)  -  n3  > 
\support(Si)  U  support(S2)\.  The  first  two  follow  directly 
from  our  assumptions  that  <5i  £  jp(Pi)  and  5 2  £  Jp(P2)- 
For  the  third  condition,  we  reason  as  follows. 

We  have  from  our  assumptions  that  7c (Ci)  A  support{5\) 
and  7c(C2)  ©  support{S2) .  Thus,  we  have 

7c(Ci)  U  7c(C2)  2  support(Si)  U  support(52) 

and  finally 

|7c(Ci)  U  7c(C<2)|  >  \support(Si)  U  support(S2)\ 
Utilizing  the  inclusion-exclusion  principle,  we  have 

l7c(Ci)|  +  |7c(C2)|  -  |7c(C,1)  n  7c(C2)| 

>  \support(Si)  U  support(52)\ 

Since  we  have  |7c(C)|  =  #((7),  we  can  rewrite  this  to  the 
following. 

#(C'1)  +  #(C'2)-|7c((71)n7c(C,2)| 

>  \support(6\)  U  support(52)\ 

It  remains  to  show  that  |7c(C'i)  (~1  7c(C2)|  =  n 3.  We 
have  that  7c(Ui  nc  C2)  =  7c(Ci)  D  7c (C2)  (that  is, 
nc  is  precise).  This  allows  us  to  complete  the  final  step, 
concluding  that  n 3,  which  is  defined  as  #(Ci  flc  C2)  is 
equal  to  |7c(C'i)  n7c(C2)|. 

Upper  Bound:  We  next  show  that  the  upper  bound 
holds.  Our  goal  is  to  show  the  following. 

Pi  ©  P2  >  \overlap(Si,S2)\ 

Expanding  our  definitions  yields  the  following  formula. 

min(s™ax,  s™ax,  713)  >  \support(8i)  (1  support(S2)\ 

We  first  note  that  the  following  holds. 

\support{5\)  A  support{S2)\  <  \support{5\)\  <  s“ax 

Thus  s"lax  is  a  sound  upper  bound.  Similarly,  we  have 

\support{5\)  A  support{S2)\  <  \support{52)\  <  s“ax 

which  shows  that  s“ax  is  a  sound  upper  bound.  Finally,  we 
note  that  our  assumptions  give  us  support^ <5i )  C  7c (Ci)  and 
support^  1)  C  7c (Ci).  Thus  we  have  the  following. 

support(Si)  IT  support(S2)  C  7c(Ci)  O  7 c(C2) 

We  showed  previously  that  7x3  =  |7c(C'i)  n7c(U2)|.  Thus 
we  have 

\support(8i)  fl  support(S2)  \  <  n 3 

which  shows  that  n 3  is  a  sound  upper  bound. 

Since  all  of  s“ax,s2lax,  and  77.3  are  sound  upper  bounds, 
their  minimum  is  also  a  sound  upper  bound.  ■ 


Lemma  60. 

\support(5\  +  62)|  = 

\support(5i)\  +  \support(82)\  —  \overlap(Si,  <)2)| 

Proof:  First  we  note  that  supported  1  +  52)  =  {0  \ 
(5i(<j)+52((t)  >  0}.  Since  the  range  of  <5i  and  S2  is  [0, 1],  we 
have  that  di(cr)  +  <)2(cr)  >  0  if  and  only  if  either  di(cr)  >  0 
or  5 2(<t)  >  0.  Thus,  we  have  a  £  support{8\  +  52)  if  and 
only  if  o  £  support{5\)  or  <7  £  support(S2),  which  implies 
support(8i  +  82)  =  support(Si)  U  support{82). 

Next,  we  note  that  for  any  sets  A,  B  we  have  |^4  U  P?|  = 
|7L|  +  \B\  —  \AnB\.  Utilizing  this  statement  with  A  = 
support(Si)  and  B  =  support(S2)  completes  the  proof.  ■ 

Lemma  12  (Soundness  of  Plus).  If  Si  £  jp(Pi)  and  S2  £ 
7p(P2)  then  5i  +  <52  G  7p(Pi  +  P2). 

Proof:  Suppose  ^  £  7p(Pi)  and  S2  £  7p(P2).  Then 


we  have  the  following. 

support^! )  C  7c(Ci)  (38) 

sfn  <  \support{8i)\  <  sf”  (39) 

mfn  <  1 1  ft  1 1  <  m“ax  (40) 

Vcr  £  support(Si).  p™"  <  5i(cr)  <  p"lax  (41) 

and 

support(S2)  C  jc(C2)  (42) 

s“in  <  \support{82)\  <  s^ax  (43) 

m™in  <  ||(52||  <  m™ax  (44) 

Vcr  e  support(S2) .  p“m  <  S2(o)  <  p2lax  (45) 


The  definition  of  abstract  plus  has  special  cases  when 
either  of  the  arguments  are  zero,  that  is,  if  iszero(Pi)  or 
iszero(P2).  Without  the  loss  of  generality,  let  us  assume 
iszero(P2 )  and  thus  by  definition  Pi  +  P2  =  Pi.  Since 
lv{P2)  =  {ODist}-  where  ODist  is  the  distribution  assigning 
probability  of  0  to  every  state.  Therefore  S2  =  Onist  and 
thus  <)i  +  S2  =  <5i .  But  we  already  have  <5i  €  7p(Pi)  by 
assumption,  hence  we  are  done  in  this  case. 

In  the  case  when  not  iszero(P±)  and  not  iszero(P2)  we 
must  show  the  following. 

support{5\  +  S2 )  C  7c(Ci  Uc  C2)  (46) 

max  {s™111  +  s“ln  —  Pi  ©  P2l  0}  <  \support(Si  +  S-2)  | 

(47) 

\support(8i  +  S2)  |  <  min  {s™ax  +  s“ax  -  Pi  ©  P2,  #(U3)} 

(48) 

mfn  +  m“in  <  ||5i  +  S2 1|  <  m“ax  +  m“ax  (49) 

We  also  must  show  the  conditions  on  pmin  and  pmax  for  the 
sum. 

Condition  (46)  follows  from  (38)  and  (42)  and  the  fact 
that  Uc  over-approximates  union.  The  key  step  is  noting 
that  support{5i  +  i52)  =  supported  1)  U  supporfS^.  To  show 
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this  we  consider  some  a  £  support(8i  +  J2).  We  have  that 
(<5i  +  <52) (cr)  >  0  which,  expanding  the  definition  of  +, 
yields  <5i (cr)  +  62 (cr)  >  0.  Since  the  range  of  (q  and  62  is 
[0,1],  this  implies  that  either  <5i (cr)  >  0  or  62(0')  >  0  and 
thus  a  £  support(8\)  or  a  £  support (£2). 

Conditions  (47)  and  (48)  follow  from  (39)  and  (43)  and 
Lemmas  59  and  60.  We  have  s“ln  <  \support(6±)\  from 
(39)  and  sf1111  <  \support(82)\  from  (43).  Monotonicity  of 
addition  then  gives  us 

s“m  +  s2nin  <  \support(Si)\  +  \support(S2)\ 

From  Lemma  59  we  have  \overlap(8i,  62)1  <  P\  ©  P2  and 
thus 

-Pi  ©  P2  <  —  \overlap(5i,  <52)| 

Combining  with  the  above  yields 

smm  +  gmi„  _  p±  ©  p2  < 

\support(8i)\  +  \support(82)\  —  \overlap(8i,  1)2)1 

We  can  then  rewrite  the  right-hand  side  according  to  Lemma 
60  to  obtain 

smin  +  smi„  _  Pl  ©  p2  <  | SUPPOrt(8i  +  S2)\ 

which  is  condition  (47). 

Condition  (48)  follows  the  same  reasoning.  We  have 
\support(Si)\  +  | support(S2)\  <  s]nax  +  s“ax  by  (39)  and 
(43).  We  then  apply  Lemma  59  and  60  to  obtain  condition 
(48). 

For  Condition  (49),  note  that 

ii<5i  +  <52||  =  +  <J2(o-))  =  X^i(CT)  +  X^2(cr) 

G  G  G 

This  is  then  equivalent  to  || <5i  ||  +  || 1| -  We  have  shown  that 
|| <5i  +  <52 1|  =  ||(>i||  +  || 1| •  Condition  (49)  then  follows  from 
monotonicity  of  addition  applied  to  (40)  and  (44) 

We  now  consider  the  pmin  and  pmax  conditions.  Let  P3  = 
Pi  +  P2  and  S3  =  (5i  +  82-  We  must  show. 

Vcr  £  support (<53)  .  p3nln  <  63(0)  <  p“ax 

The  values  p™m  and  p™ax  are  defined  by  cases  and  we 
consider  these  cases  separately.  In  one  case,  we  have  that 
pimn  sum  js  min(p“m ,  p™m).  This  is  always  a  sound 
choice.  To  see  why,  suppose  a  £  support(8i  +  82).  Then 
a  £  support(Si)  or  a  £  support(82).  If  cr  G  support(5 1), 
then  (<)i  +  <52)(cr)  =  i5i(cr)+<52(cr)  is  at  least  p“ln.  Similarly, 
if  cr  £  support (62)  then  (<5i  +  <52)(cr)  >  <52(ct). 

Similarly,  the  value  p™ax+p™ax  is  always  a  sound  choice 
for  p“ax.  Consider  cr  £  support(S3).  Then  cr  £  support(Si) 
or  cr  G  support(S2)-  If  cr  G  support(Si)  and  cr  ^  support(S2), 
then  we  have 

S3  (cr)  =  c5i(cr)  +  62(a)  =  <5i(cr) 

By  (41)  we  then  have  J3(cr)  <  p™ax  and  thus  53(cr)  < 

pmax  _|_  pmax  as 


Similarly,  if  a  ^  support(Si)  and  a  £  support(S2)  then 
by  (45)  we  have 

U3 (a)  =  02(a)  <  p2  <  Pi  +  p2 

Finally,  if  a  £  support(Si)  and  a  £  support (<52)  then  by 
(41)  we  have  <5i(cr)  <  p“ax.  By  (45)  we  have  62(a)  <  p“ax. 
Combining  these  we  have  <5i (cr)  +  82(a)  <  p™ax  +  p“ax 
which  is  equivalent  to  63(a)  <  p™ax  as  desired. 

Next  we  consider  the  Pi  ©  P2  =  #(C3)  case  for  p“ln. 
We  must  show  that  p™m  +  p“ln  is  a  sound  lower  bound 
on  83(a)  for  a  £  support (^3).  We  have  by  Lemma  59  that 
Pi  ©  P2  <  \overlap(8i,82)\-  Since  P3  ©  P2  =  #(C3) 
and  #(C3)  >  \overlap(8i,82)\,  we  have  that  #(C3)  = 
\overlap(8i ,  <52)  | .  Expanding  the  definition  of  overlap (81, 82) 
yields 

\support(8i)  n  support (<52)|  =  #(C3)  (50) 

We  have  from  (46)  that  support(8i  +  (>2)  C  ^(^3)  and 
from  the  proof  of  (46)  we  have  that  support(8i  +  d'2)  = 
support(Si)  U  support(82)-  Combining  these  yields 

\support(Si)  U  support (<52)|  <  #(C3) 

Combining  this  with  (50)  yields 

\support(8i)  U  support(82)\  <  \support(Si)  IT  support(82)\ 

For  any  sets  A,  B,  we  have  that  \  A  U  B\  >  \A  D  B\  and  thus 
the  above  inequality  implies  the  following. 

\support(8i)  U  support(82)\  =  \support(8i)  n  support(82)\ 

The  fact  that  the  size  of  the  intersection  and  the  size  of  the 
union  of  support(Si)  and  support ((52)  is  identical  implies 
that  support(Si)  =  support (<52).  This  implies  that  for  all  cr, 
we  have  a  £  support(8 1)  if  and  only  if  a  G  support(82). 

Now  consider  a  £  support (<53).  We  have  a  £  support(Si) 
or  a  £  support (62),  as  before,  but  now  we  can  strengthen 
this  to  a  £  support(Si)  and  a  £  support (<52).  By  (41)  we 
have  p™ln  <  81(a)  and  by  (45)  we  have  p“ln  <  82(a). 
Thus  we  have 

pfn  +  pfn<^iM  +  ^(a) 
which  was  our  goal. 

Finally  we  consider  the  P3  ©  P2  =  0  case  for  p"iax 
(the  “otherwise”  case  in  Definition  11).  Consider  a  a  £ 
support(S3).  We  must  show  that  J3(cr)  <  max(p5nax,  p™ax). 
We  have  that  either  a  £  support(Si)  or  a  £  support(82) ■  We 
cannot  have  both  since  P3  ©  P2  =  0  which,  by  Lemma  59 
implies  that  \overlap(8i,  <52)|  =  0.  If  a  £  support(Si)  then 
by  (41)  we  have  <5i (cr)  <  p”lax.  We  have  a  ^  support^ ) 
and  thus  82(a)  =  0.  Thus  we  reason  that 

61(a)  +  s2(a)  =  81(a)  <  Prx  <  max(Prx,prx) 

Similarly,  if  a  £  support(82)  then  we  apply  (45)  to  obtain 
81(a)  +  82(a)  =  82(a)  <  Prx  <  max(Prx,prx) 
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D.  Product 

Lemma  13  (Soundness  of  Product).  If  S\  G  Jp(-Pi)  and 
S 2  G  'yp(P2)  then  Si  x  S2  G  q fp(Pi  x  P2). 

Proof:  By  assumption,  we  have  the  following  for  i  = 

1,2. 


support  (Si)  C  7c(C,;)  (51) 

s”in  <  |sHpport(<yi)|  <8““  (52) 

mfn<  H^ll  <m“ax  (53) 

Vrr  G  support(Si)  .  p™11  <  5(<Ji)  <  p™ax  (54) 

Let  S3  <5|  x  6 2  and  P:i  =  l\  x  If.  Recall  the  definition 
of  P3. 
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Cl  x  C2 

—.min 

P3 

_  —min  —min 

—  Pi  •  P2 

—max  _ 

P3  — 

—max 

Pi 

—  max 

P2 

0min 

b3 

_  cmin  0min 

~  bl  '  b2 

b3  — 

0max 

bl 

0max 

b2 

—-.min 

m3 

_  —.min  —.min 

—  111 1  ’  m2 

—.max  _ 

m3  — 

—.max 

111  i 

—.max 

’  m2 

We  must  show  the  following  four  claims. 


support(S. 3)  C  7c  (C3)  (55) 

sfa  <  \support(S3)\  <  s?™  (56) 

mfn  <  \\S3 1|  <  m“ax  (57) 

Ver  G  support (S:i)  .  p“ln  <  S3(o)  <  p“ax  (58) 


Also,  recall  the  definition  of  concrete  product. 

£1  x  5-2  =  AOi,ct2)-  <5i(cti)  •  62(02) 


Let  Vi  =  j\’(5i)  and  V2  =  fv(S2). 

Claim  (55)  -  Support.  Let  a  =  (01,02)  G  support(S3). 
Thus  it  must  be  that  81(01)  >  0  and  S2(o2)  >  0,  thus,  by 
(51),  0 1  G  support(Si)  C  7c (Ci)  and  cr2  G  support(S2)  C 
7c(C2),  therefore  a  G  7c(<53)-  □ 

Claim  (56)  -  Support  points.  Using  (52)  we  get  the 
following. 

sfn  •  s“in  <  \support(61)\  ■  \support(S2)\  <  s“ax  •  s“ax 


Likewise,  the  size  of  support(S3)  can  be  equated  as 
follows. 


\support(S3)\ 


|(CTl,(J2) 


0  1  G  support(Si), 
a 2  G  support(S2) 


\support(Si)\  ■  \support(S2)\ 


This  completes  the  claim  as  s“m  =  and  s“ax  = 

gmax  #  gmax  |~ | 


Claim  (57)  -  Mass. 

INI  =  E 

cr  (^support  (83) 

=  E  81(01)  ■  82(02) 

(cri  ,c72)Gsupport(83) 

E  (  E  ^(ctt)  •  62(02)  J 

aiEsupport(Si)  \a2^support(82)  J 

E  5i (cti)  E 

cr\(ziSupport(8\)  a2^support(S2) 

-  E  •  INI 

<j\ ^support{8\ ) 

=  INI-INI 

□ 

Likewise,  by  (53),  we  have  the  following. 

mfn  •  m“in  <  INI  •  INI  <  m“aX  •  m”aX 

This  completes  the  claim  as  m“ln  =  m™ln  •  m™111  and 

^max  _  y^max  — .  max 

m3  —  ml  '  m2 

Claim  (58)  -  Probability.  Let  a  =  (01,02)  € 

support(S3).  Thus  o\  G  support(Si)  and  o2  G  support(d2). 
Also,  S3  (a)  =  5i(cti)  •  <52(ct2).  By  (54),  we  have  p“ln  < 
<^i(<Ji)  <  p“ax  and  p™m  <  82(02)  <  p™ax.  Therefore 

P3  =  Pi  P2  —  63(0)  S  Pi  'P2  =  P3 

This  completes  the  claim  and  the  proof.  □ 


E.  Conditioning 

Definition  61.  Given  a  set  of  states  S  and  a  boolean 
expression  B,  let  S\B  be  the  subset  of  S  that  satisfy  the 
condition  B  and  S\B  be  the  subset  of  S  that  do  not  satisfy 
the  condition.  Formally, 

S\B  =  {0  G  S  |  lB]a  =  true  } 

S\B  =  {0  G  S  |  \B\a  =  false  } 

Lemma  15  (Soundness  of  Conditioning).  If  6  G  'yp(P)  then 
8\B  G  7p(P  I  B). 

Proof:  Let  S2  =  8\B.  Recall  the  definition  of  the 
conditional  distribution: 


8\B  =  A 0.  if  [P]cr  then  5(a)  else  0 

Let  P2  =  P  |  B.  The  construction  of  P2  produces  the 
following  parameters. 

pmin  _  pmin  gmin  =  maX  { Smin  -  U,  0 } 

„max  nmax  cmax  — .•  Jomax  ,r) \ 

P2  — P  b2  —  111111  tb 

m™'"  =  max  {p2lin  •  s^in,  mmin  -  pmax  •  min  {smax,  n}} 
m“ax  =  min  {p“ax  •  s^ax,  rnmax  -  pmin  •  max  {smin  -  n,  0}  } 

C2  =((B))C 
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The  quantities  n  and  n  are  defined  in  such  a  way  that 
n  over-approximates  the  number  of  support  points  of  6 
that  satisfy  B,  whereas  n  over-approximates  the  number  of 
support  points  of  5  that  do  not  satisfy  B.  Also,  ((B))  C  is 
defined  to  contain  at  least  the  points  in  C  that  satisfy  B. 
Making  these  properties  precise  gives  us  the  following. 


\support(8)\B\  <  n  (59) 

\support(5)\B\  <n  (60) 

7c(C)|BC7c(«B»C)  (61) 

By  assumption  we  have  the  following. 

supported)  C  7c  (C)  (62) 

smin  <  \support(5)\  <smax  (63) 

mmin  <  ||5||  <  rnmax  (64) 

Vcr  G  support(S)  .  pmin  <  8(a)  <  pmax  (65) 

We  need  to  show  the  following  four  claims. 

support(5 2)  C  7c(C2)  (66) 

s“in  <  \support(82)\  <  s^ax  (67) 

m“in<  ||#2||  <m“ax  (68) 

Vcr  €  support(52)  ■  p™n  <  52(a)  <  p“ax  (69) 

Claim  (66)  -  Support.  Let  a  G  support(52).  Thus  it 
must  be  that  a  €  support(S)  and  \B\a  =  true  .  By 
(62),  we  have  cr  G  7c(C),  therefore  a  G  Jc(C2)  as 
{a  G  7c(C)  I  \B\a  =  true  }  C  7c (C2)  by  construction  of 
C2.  □ 


Claim  (67)  -  Support  points.  Let  us  write  support(S)  as 
a  union  of  two  disjoint  sets. 

support(S)  =  support(5)\B  U  support(5)\B 

Given  the  disjointness  of  the  two,  we  also  have  the 
following. 

\support(6)\  =  \support(5)\B\  +  \support(5) \B  | 

Now  note  that  support(52)  =  support(5)\B .  Thus  we  can 
write  \support(52)\  =  \support(8)\  —  \support(5)\B\.  We 
can  therefore  estimate  the  size  of  the  support  of  62  in  the 
following  manner. 

\support(S2)\  =  \support(8)  \  —  \support(5)\B\ 

<  \support(5)\ 

<  smax  [  by  (63)  ] 

Therefore,  using  (59)  and  the  above,  we  have 
\support(52)\  <  min{smax,n}  =  s™ax. 

Going  in  the  other  direction,  we  can  write  as  follows. 

\support(S2)\  =  \support(8)\  —  \support(5)\B\ 

>  smm  —  \support(5)\B\  [  by  (63)  ] 

>  smin  -  n  [by  (60)  ] 


Since  all  sets  are  trivially  of  size  at  least  0,  we  have 
\support(52)  |  >  max  {smin  —  n,  0}  =  s™111.  □ 

Claim  (69)  -  Probability.  Note  that  we  will  show  the 
probability  claim  before  the  mass  as  we  will  use  the  truth 
of  the  probability  claim  in  the  mass  arguments. 

Let  a  G  support(52) .  By  definition  of  52,  we  have  52(a)  = 
8(a).  Thus  a  G  support(5)  so  by  (65)  we  have: 

pmi„  =  pmi„  <  =  <  pmax  =  pmax 

□ 

Claim  (68)  -  Mass.  Let  us  first  show  the  following  bound 
on  the  size  of  support(5)\B . 

max{smin  —  n,  0}  <  \support(8)\B\  <  min{smax,n} 

(70) 

Since  \support(8)\  =  \support(8)\B\+  \support(5)\B\,  we 
can  say  \support(8)\B\  =  \support(5)\  —  \support(5)\B\  and 
continue  to  the  bound  in  the  following  manner. 

\support(8)\B\  =  \support(5)\  —  \support(S)\B\ 

>  smm  —  \support(5)\B\  [  by  (63)  ] 

>  smin  -  n  [by  (59)  ] 

Therefore  \support(5)\B\  >  max{smin  —  n,  0}  as 
claimed.  For  the  other  end  of  the  inequality,  note  that  we 
have  \support(8)\B\  <  \support(8)\  <  smax  by  (63).  Also, 
by  (60),  \support(5)\B\  <  n.  Therefore  \support(5)\B\  < 
max  {smax,n},  completing  our  bound. 

Now,  let  us  write  ||<5||  in  two  parts. 

11*11  =  E  %) 

a(zSupport(8 ) 

E  +  E 

cr  (^support  (8)\B  crG  support  (<5)|5 

=  11*211+  E 

<r(zSupport(8 )  |  B 

Therefore  \\82\\  =  ||*||  -  EaGSuppor,(S)\B 


ll*2||  = 

11*11  - 

E  ^(<t) 

a(zSupport(8)  |  B 

< 

rnmax 

E  <5(£t) 

[by 

(64)] 

a  ^support (8)  \  B 

< 

mmax 

-  E  pmin 

[by 

(65)] 

a  ^support (8)  \  B 

= 

rnmax 

-  \support(8)\B\  ■  pmin 

< 

rnmax 

-  max  {smin  -  n,  0}  •  pmin 

[by 

(70)] 
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Also,  we  can  bound  the  mass  using  our  other  already  Let  S2  =  p  ■  8\  and  P2  =  p  ■  Pi.  Let  us  assume  that  p  f  0. 
proven  conditions.  In  this  case  we  need  to  show  the  following. 


Nl  =  E  w 

cr£support(52 ) 

<  E  P”ax  [  by  (69)  ] 

cr(zSupport(52 ) 

=  \support(S2)\  ■  p“ax 

<  s^ax  •  p^ax  [  by  (67)  ] 

Combining  the  bounds,  we  have  half  of  our  probability 
condition. 

INI  <m“ax 

=  min  {p“ax  •  s2nax,  mmax  -  pmin  •  max  {smin  -  n,  0}  } 

For  the  other  half,  we  proceed  similarly. 

ni  =  11*11-  E  j (<j) 

<j(zSupport(8)  |  B 

>  rnmm  —  E  d(cr)  [  by  (64)  ] 

a  £  support  (<5)  |  B 

>  mlnin  -  E  PmaX  [  by  (65)  ] 

a  £  support  (<5)  |  B 

=  rnmin  -  \support{5)\B\  ■  pmax 

>  rnmin  -  min  {smax,  n}  •  pmax  [  by  (70)  ] 


support (<5i)  =  support (S2)  C  7c(C'2)  =  7c(Ci) 

smm  =  smin  <  \support^2)\  =  \support{S2)\  <  8%**  =  S?laX 

p  ■  mfn  =  m“in  <  ||5i  ||  <  m“ax  =  p  •  m“ax 
Vcr  G  support(S2)  ■ 

P'Pl  =  P2  <<J2(cr)<p2  =p-  Pi 

The  first  two  conditions  are  trivially  satisfied  given  the 
lack  of  change  in  the  various  parameters.  For  the  mass 
condition,  note  that  ||J2||  =  =  H2aP  '  lMcr)  = 

p  ■  ||i5i||.  The  probability  condition  is  also  trivially  satisfied 
as  S2(a)  =  p  ■  <5i(ct). 

In  the  case  that  p  =  0,  the  abstract  scalar  product  is 
defined  with  s!fin  =  s^ax  =  p(fin  =  p^ax  =  mr2nin  = 
m™ax  =  0  and  C2  =  0c-  In  this  case  note  that  support(S2)  = 
0  =  7c  (0c).  and  thus  the  conditions  hold  trivially.  ■ 

G.  Uniform 

Lemma  62  (Soundness  of  Uniform).  If  5  G  jp(P )  and  S  = 
uniform  x  n i  n2  then  |5]<5  G  7p (((-S'))  P ’). 

Proof:  Recall  the  semantics  of  the  statement. 

[uniform  x  ni  ?r2]<5  =  (<5  [  fv(S)  —  {x})  x  S2 

The  distribution  52  is  defined  as  follows. 


And  likewise  another  bound  using  our  other  conditions. 

INI=  E  w 

a£support(52 ) 

>  E  P2nin  t  by  (69)  ] 

cr£support(52 ) 

=  \support(S2)  |  •  p2nin 

>  s™n  •  pfn  [  by  (67)  ] 

Combining  the  two  bounds,  we  have  the  final  element  of 
our  proof. 

INI  >  m“in 

=  max  {p“in  •  sfn,  rnmin  -  pmax  •  min  {smax,  n}} 

□ 

■ 

F.  Scalar  product 

Lemma  17.  If  Si  G  7p(Pi)  then  p  ■  <5i  G  7p(p  •  Pi). 

Proof:  By  assumption  we  have  the  following. 
support(61)  C  7c (Ci) 
sfn  <  \support{Sx)\  <  s“ax 
mf "  <  lift  ||  <  m“ax 
Vcr  G  support^)  .  pf 1  <  5\{a)  <  pj”“ 


S2  =  Act.  if  m  <  cr(x)  <  n2  then  -  else  0 

n2  —  tti  +  1 

The  abstract  semantics  are  similar. 

((uniform  x  n\  n2))  P  =  (fx(P))  x  P2 

Here  P2  is  defined  with  p™ln  =  p™ax  =  no_lll+1, 
s2lin  =  s(fax  =  n2  -  m  +  1,  m™in  =  m2lax  =  u'and 
C2  =  ({x  >  m,  x  <  n2}  ,  {x}). 

By  construction,  we  have  S2  G  P2  thus  the  lemma 
follows  from  Lemma  7  (Soundness  of  Forget)  and  Lemma 
13  (Soundness  of  Product).  ■ 

II.  While  loops 

Definition  63.  First  we  have  some  preliminary  definitions. 
Given  some  set  of  variables,  we  have  the  following,  where 
each  distribution  or  state  in  each  statement  is  understood  to 
be  defined  over  the  same  set  of  variables. 

•  Two  distributions  are  ordered ,  or  h  <  ^2  iff  for  every 
state  a,  Si(a)  <  S2(a). 

•  Two  probabilistic  polyhedra  are  ordered ,  or  p  Cp  P2 
iff  for  every  S-\  G  jp(Pi),  there  exists  S2  G  7p(P2)  with 
<Al  S  ^2- 

•  The  zero  distribution  6  is  the  unique  distribution  with 
S(a)  =  0  for  every  a.  We  will  use  Onist  to  refer  to  this 
distribution. 
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•  A  zero  probabilistic  polyhedron  P,  or  iszero(Pi)  is  one 
whose  concretization  contains  only  the  zero  distribu¬ 
tion,  that  is  7p(P)  =  {0Dist}- 

Lemma  64.  Let  P,;  be  consistent  probabilistic  polyhedra, 
that  is,  "fp(Pi)  b  0.  Then,  P\  +  P-2  Ep  P1  iff  iszero{P2). 

Proof:  In  the  forward  direction,  we  have  P1+P2  Ep  Pi. 
Now,  let  us  consider  a  P2  with  not  iszero{P2).  Thus  there 
is  62  £  >(P2)  with  ||^2 1|  >  0.  Let  Si  £  7p(Pi)  be  the 
distribution  in  7p(Pi)  maximizing  mass,  that  is  ||<5i||  >  ||/[  || 
for  every  £  7p(Pi).  By  Lemma  12,  61+62  £  7p(Pl  +  P2) 
and  by  the  definition  of  P3  +  P2  Ep  Pi.  there  must  be  <J3  £ 
7p(Pi)  with  Si  +  S2  <  <53-  Thus  ||<53||  >  || Ai  +  <52||  =  ||<5i||  + 

| l^l  >  ||<5i||.  This  contradicts  that  b  was  mass  maximizing 
in  7p(Pi). 

In  the  backward  direction,  our  definition  of  abstract  plus 
makes  I\  +  P2  identical  to  l  \ .  Thus  Pi  +  P2  =  Pl  Ep  Pi¬ 
rn 

Definition  65.  Given  a  statement  S  =  while  B  do  S',  a 
distribution  S  and  a  probabilistic  polyhedron  P,  let  us  define 
a  few  useful  items. 

.  u(f)^\6.f(lS'mB))  +  6hB 

.  Si  =6 

.  Si+i  =  [S"](b|P) 

.  AndEfEr=i(^hP) 

•  J-Dist  is  the  function  that  takes  in  any  distribution  and 
produces  the  zero  distribution  Onist.  that  is  J-Distb)  = 

ODist- 

Similarly  we  have  the  abstract  versions  of  the  definitions. 

.  f 2(P)  =  A P.  F(((S'))  (P\B))  +  P\  ~^B 

.  P3  =  P 

.  Pi+i  =  ((S'))  (P  |  B) 

•  =  E”=l  (P  I  ~^B) 

•  _Lp  is  a  function  that  takes  in  any  probabilistic  polyhe¬ 
dron  and  produces  a  zero  probabilistic  polyhedron,  that 
is  iszero(+p(P))  for  every  P. 

The  semantics  of  while  loops  are  defined  as  such: 

[S']  =  [while  B  do  S']  =  lfp(w) 

«S))  =  ((while  B  do  S'))  =  lfp(ft) 

While  such  definitions  are  of  theoretical  interest,  they 
are  not  particularly  useful  for  implementations,  given  our 
lack  of  a  widening  operator.  Thus,  our  security  checks 
will  always  be  conditioned  on  termination  of  the  abstract 
interpretation,  defined  below.  We  show  that  termination  of 
the  abstract  interpretation  implies  termination  of  all  corre¬ 
sponding  concrete  executions.  This  is  crucial,  as  our  concrete 
semantics  (due  to  Clarkson  et  al.  [9])  assumes  termination 
to  avoid  leaks.  To  make  this  termination  condition  explicit, 
we  provide  an  alternate  concrete  semantics  for  terminating 
while  loops  and  show  that  this  gives  results  equivalent  to 
those  of  the  original  semantics. 


Definition  66.  The  termination  of  [5]  S  is  defined  as  follows. 

•  If  S  is  an  elementary  statement  (assignment,  skip, 
uniform),  then  [S'] S  terminates. 

•  If  S  is  a  sequence,  if  statement,  or  a  probabilistic 
choice  statement,  then  [S']  <5  terminates  iff  the  various 
evaluations  steps  to  evaluate  S  terminate.  This  depends 
on  the  statement  type,  for  S  =  S|  ;  S2,  for  example,  it 
means  that  [S3] <5  terminates  and  so  does  [S2]  ( [Si] <5) . 

•  If  S  =  while  B  do  Si  is  a  while  statement,  then  [S]<5 
terminates  iff  there  exists  n  with  Sn  =  Onist  and  the 
evaluation  steps  as  per  definition  of  Si  terminate  for  all 
i  up  to  n. 

The  termination  of  ((S))  P  is  framed  similarly,  except  in 
the  while  case,  we  require  the  existence  of  n  with  iszero(Pn ) 
and  the  termination  of  the  abstract  evaluations  as  in  the 
definitions  of  Pi  for  all  i  up  to  n. 

The  A,  and  <I\  capture  exactly  the  concrete  and  abstract 
values  when  termination  is  assumed. 

w1  (J-Dist)  (£)  = 

=  Ar 

w2 (J-Dist) (i5)  =  ([S'PIP))  b B  +  SffB 
=  $2\^B  +  5i\-*B 
=  A2 

(J-Dist)  (<5)  =  tP-b^DistHI.S'HlP)  +  SffB 
=  Aj_i  +  SffB 

=  At 

Likewise  fL(J_p)(P)  =  <!>.;. 

Definition  67.  Terminating  semantics  of  while  loops  are  as 
follows. 

[while  B  do  Si]#  =  An 

Where  n  is  the  least  index  with  5n  =  Onist-  Likewise  for  the 
abstract  case. 

((while  P  do  S'i))P=  “Tn 

Where  n  is  the  least  index  with  iszero(Pn). 

Lemma  68.  //[while  B  do  S/M  is  terminating,  then  An  = 
(lfp(tu))  (/),  noting  that  lfp(w)  is  the  original  semantics  of 
a  while  loop. 

Proof:  As  noted  in  [16],  the  evaluation  of  a  while  loop 
on  a  distribution  is  equal  to  an  infinite  sum: 

OO 

lsjS  =  J2^hB 

i— 1 

By  the  termination  assumption  we  have  an  n  with  5n  = 
Onist ■  Now,  since  b+i  =  [<Sl,]b|.B  hence  the  mass  of  b+ 1 
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cannot  exceed  the  mass  of  St,  it  is  the  case  that  if  Sn  =  Onist. 
then  Si  =  Ooist  for  every  i  >  n.  Thus  the  infinite  sum  above 
can  be  shortened. 

OO 

lsjS  =  J2^hB 

i—  1 
n 

=  +  Ooist 

i- 1 


Remark  69  (Nature  of  Termination).  If  ((S'))  P  terminates, 
then  so  must  the  evaluation  of  all  of  its  components  as  de¬ 
fined  by  the  semantics.  This  is  immediate  from  the  definition 
of  termination. 

I.  Soundness  of  Abstraction 

Theorem  6.  For  all  P ,  S,  if  S  G  Jp(P)  and  ((S))  P  termi¬ 
nates,  then  [S]<5  terminates  and  [S]<5  G  7p(((S))  P). 

Proof:  Let  us  show  this  by  stmctural  induction  on  S. 
As  base  cases  we  have  the  following. 

•  S  =  skip.  In  this  case  we  have  [S] <5  =  S  and  ((S))  P  = 
P.  Termination  is  not  an  issue  and  the  claim  holds  by 
assumption. 

•  S  =  x  :=  E.  Here  non-termination  is  also  not  a 
possibility  given  non-recursive  definition  of  assignment. 
Also,  by  Lemma  9  (Soundness  of  Assignment)  we  have 

[SpG>(((S))P). 

•  S  =  uniform  x  n i  ri2-  Again,  there  is  no  termination 
issues  and  the  claim  follows  from  Lemma  62  (Sound¬ 
ness  of  Uniform). 

Let  us  thus  assume  the  claim  for  sub-statements  of  S  and 
show  it  for  S  itself.  Note  that  the  inductive  assumption  is 
general  for  all  6,  P  with  6  G  Jp(P).  S  has  several  cases. 

•  S  =  Si  ;  1S2.  By  the  termination  remark,  we  know 
((Si))  P  terminates  and  thus  by  induction  [Si](5  termi¬ 
nates  and  is  in  7p(((Si))P).  We  then  apply  induction 
once  more  with  S2  to  find  that  [S2]([Si]<5)  =  [S]<5 
terminates  and  is  in  7p(((S2))  (((Si))  P))  =  7p (((S))  P). 

•  S  =  if  B  then  Si  else  S2.  By  the  termination  re¬ 
mark,  we  know  that  ((Si))  (P  |  B)  and  ((S2))  (P  |  -<B) 
terminate.  By  Lemma  15  (Soundness  of  Conditional) 
we  have  5\B  G  7p(P  |  B)  and  S\^B  G  7p(P  |  ~<B). 
We  thus  apply  induction  to  both  sub-statements  to  con¬ 
clude  that  [Si]((5|f?)  and  HS2] (<5|  — both  terminate 
and  are  in  7p(((Si))  (P  |  B))  and  7p(((S2))  (P  |  ~^B)) 
respectively.  Finally  we  apply  Lemma  12  (Soundness 
of  Plus)  to  conclude  [S]<5  =  |Si](5|B)  +  [S2](5|-.B)  G 
7p(«S1))  (P  I  B)  +  ((S2))  (P  |  -P))  =  7p(«S))  P). 


•  S  =  pif  p  then  Si  else  S2.  This  case  is  identical  to 
the  previous  except  we  use  Lemma  17  (Soundness  of 
Scalar  Product)  in  place  of  Lemma  15  (Soundness  of 
Conditional). 

•  S  =  while  B  do  Si. 

For  this  last  case  we  must  first  show  a  claim.  For  every  S', 
P'  with  S'  G  7p(P'),  and  every  i  we  have  the  following. 

S'  G  Jr  {Pi)  (71) 

A'  G  >(<&')  (72) 

Let  us  show  this  claim  by  induction  on  i.  As  the  base  case 
we  have  S{  =  S'  and  A[  =  S[\^B  =  S'\^B.  Also  P(  =  P' 
and  fI>'|  =  P{  |  —B  =  P'  |  — /i.  By  assumption  we  had 
S'  G  7p  (P')  so  the  first  part  of  our  claim  holds  trivially.  For 
the  other  we  apply  Lemma  15  (Soundness  of  Conditional) 
to  conclude  A(  G  7p(<f>,i). 

Let  us  assume  the  claim  holds  for  all  i  <  n  and  show  that 
it  holds  for  n. 

We  have,  by  definition,  S'n  =  [Si] {S'n_i\B)  and  P'n  = 
((Si))  (Ph-i  |  B).  By  the  (inner)  induction  assumption,  we 
have  S'n_1  G  Jv{Ph-\)  so  by  Lemma  15  we  have  S’n_1\B  G 
Jv{Pn-i  I  S).  Since  ((S))  P  terminates,  then  so  must 
((Si))  P'n_  1  |  B  by  the  termination  remark.  Thus,  by  the 
(outer)  induction  hypothesis,  we  know  that  [Si]  (S'n_1  \B)  = 
S'n  G  >(((Si))  (p;_!  I  B))  =  Jv{P'n). 

For  the  second  part  of  the  claim,  we  have  A),  =  A),^,  + 
S'n\~^B  and  &n  =  <f))l_i  +  P'n  \  ^B.  By  (inner)  induction 
we  know  A(l_1  G  7p(<f)(l_i).  By  the  first  part  of  the  claim 
above  we  know  S'n  G  Jv{Ph)  so  by  Lemma  15  (Soundness 
of  Conditional)  we  have  S'n\^B  G  jv (P()  |  ~^B).  Now  we 
apply  Lemma  12  (Soundness  of  Plus)  to  conclude  A)  = 
A'n-1+S'nhB  G  JriK-1  +  Pn  I  ^B)  =  >(^).  finishing 
the  claim. 

Now,  since  ((S))  P1  terminates,  it  must  be  that  ((S))  P'  = 
A>'n  for  some  n,  according  to  the  terminating  semantics. 
Furthermore  we  have  the  following,  also  by  definition  of 
termination. 

iszero{P'n  \  ~^B)  (73) 

This  is  the  case  since  iszero{P!n)  and  the  fact  that  the 
conditioning  operation  preserves  iszero{-). 

Therefore  by  (71)  we  can  conclude  that  Sn  =  Ooist  as 
7c(Pn)  =  {Ooist}-  Therefore  [S]<5  terminates  and  by  Lemma 
68  we  have  [S']  <5  =  An.  The  issue  of  whether  n  is  the  least 
index  with  8n  =  Opist  is  irrelevant  as  if  it  were  not,  the  larger 
sum  includes  only  additional  Onist  terms.  By  (72),  we  have 
An  G  7p(<l>n)  and  we  are  done  as  <!>„  =  ((S))  P  according 
to  the  terminating  semantics.  ■ 

J.  Normalization 

Lemma  19.  If  c)i  G  7p(Pl)  then  normal  (Si)  G 

jp{normal{Pi)). 
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Proof:  By  assumption  we  have  the  following. 
support (<5i)  C  7c (Ci) 
sfn  <  \support(8i)\  <  srx 
mfn  <  ||5i||  <  m“ax 
Vtr  €  support(8\)  .  p™111  <  81(a)  <  p™ax 

If  || <5i ||  =  0  then  normal^)  is  undefined.  Since  m™111  < 
||5i||,  it  must  be  that  m™111  =  0  as  well,  and  thus  normal(Pi) 
is  likewise  undefined. 

Let  us  now  assume  ||<5i||  >  0.  Let  82  =  normal(<5i)  and 
P2  =  normal  (Pi).  We  have  two  sub-cases,  either  m™1”  =  0 
or  1  n y 1 1 11  >  0.  In  the  first  sub  case,  P2  is  defined  as  follows. 


—.min 

P2 

—.max  _  1 

P2  —  1 


=  pfin/mf 


=  1 


cmin  _  0min 

b2  —  b1 

cmax  _  cmax 

b2  —  »i 

c2  =  Ci 


Since  support (<j2)  =  support(8 1),  it  must  be  that 
support(82)  C  7c(C2)  as  C2  =  C\.  Likewise,  the  number  of 
support  point  is  is  unchanged  in  both  the  concrete  operation 
and  the  abstract  one,  hence  the  number  of  support  points 
condition  for  soundness  are  satisfied  as  well.  Also,  the 
probability  per  point  in  any  distribution  does  not  exceed 
1  hence  the  p“ax  condition  is  satisfied.  As  for  p™ln,  note 
that  if  a  £  support(82)  =  support(8\ ),  we  have  82(a)  = 
<5i(o-)/||c>i||  >  p5nm/||^i||  >  pf'H/mf8*,  by  assumption. 
Finally,  ||<52||  =  1  hence  the  m™ln  and  m“ax  conditions  are 
satisfied. 

In  the  other  case,  we  have  p”lln  >  0.  Here  P2  is  defined 
as  follows. 


—min 

P2 

_  —min 

—  Pi 

/mmax 

amin 

b2 

_  cmin 

— 

— .  mn.x 

P2 

_  —max  /—-.mm 

—  px  /!% 

n  max 
b2 

_  n  max 

— 

--min 

m2 

=  m“ax  =  1 

c2 

=  Ci 

The  support,  support  points,  total  mass,  and  p“m  con¬ 
ditions  are  satisfied  for  the  same  reason  as  in  the  previous 
case.  For  p“ax,  let  a  £  support (82)  =  support  (81 )  and  we 
have  the  following. 


<  Prx/m' 


min 

1 


K.  Security 

Before  we  prove  the  security  theorem,  let  us  show  that 
the  definition  of  abstract  conditioning  on  a  state  is  sound. 

Lemma  70.  If  8  £  'yr(P)  and  ay  £  Statey  with  V  Cfv(8) 
then  8\ay  €  7p(P  |  ay) 

Proof:  Recall  the  definition  of  P  |  ay. 

P  \  av  =  P  \  B 


With  B  =  f\xeV  (x  =  ay(x)).  Let  us  show  that  8\ay  = 
8\B,  the  rest  will  follow  from  Lemma  15. 

The  definition  of  8\ay  is  as  follows. 

8\a  =  A  a.  if  a  \  V  =  ay  then  8(a)  else  0 

Meanwhile,  8\B  is  defined  as  follows. 

8\B  =  A  a.  if  \B\a  =  true  then  8(a)  else  0 

The  correspondence  is  immediate  as  \B\a  =  true  if 
and  only  if  a  \  V  =  ay  as  per  construction  of  B.  ■ 

Theorem  22.  Let  8  be  an  attacker’s  initial  belief.  If  8  £ 
7p(P)  and  tsecuret.(S,P),  then  S  is  threshold  secure  for 
threshold  t  when  evaluated  with  initial  belief  8. 

Proof:  Let  us  consider  the  contrapositive.  That  is, 
assuming  8  £  'yp(P),  if  S  is  not  threshold  secure  for  t  and 
initial  belief  8,  then  it  is  not  the  case  that  tsecuret(S ,  P) . 

Let  82  =  [S']  8  and  83  =  82  f  L.  Since  S  is  not 
secure,  we  have  a l  £  support(83)  and  a'H  £  State# 
with  (normal((52|cri)  )  H))(a'H)  >  t.  This  implies  that 
(82\aL)  \  H  0Dist  and  therefore  S2\aL  0Dist  as 
projection  preserves  mass. 

If  ((S))  P  is  not  terminating,  then  we  are  done  as  termi¬ 
nation  is  a  condition  for  tsecuret(S,P).  So  let  us  assume 
((S))  P  is  terminating.  Let  P2  =  (( S ))  P.  By  Theorem  6,  we 
have  82  £  7p (P2).  By  Lemma  70,  S2\aL  £  7p(P2  |  aL). 
Therefore  not  iszero(P  \  a  if  as  82  \  a  1  Ouist-  Continuing, 
by  Lemma  44,  (t>2|crz,)  f  H  £  7p((P2  |  aL)  f  H)  and 
finally,  by  Lemma  19,  we  have  normal ((<52|ctl)  f  H)  £ 
7p (normal ((P2  |  aL)  \  H)).  Let  84  =  normal((52|crL)  \  H) 
and  P4  =  normal((P2  |  ol)  \  H).  Since  a'H  £  support (8 4), 
we  have  ^(cr#)  <  pf1^.  Since  84(0^)  >  t,  we  have 

t<  Pyax. 

Also,  let  P3  =  P2  I"  L.  By  Lemma  44,  we  have  £3  £ 
7p(P3)  so  ol  £  7c(C3).  We  already  had  that  not  iszero(P  \ 
ai,)  above.  Thus  ctl  is  indeed  the  witness  to  the  failure  of 

tsecuretfS,  Pf.  ■ 


Appendix  E. 

Soundness  proofs  for  Pn  (P) 

A.  Useful  Lemmas 

We  begin  with  some  lemmas  that  give  properties  of 
the  concretization  function  for  powersets  of  probabilistic 
polyhedra  and  addition  on  sets. 

Lemma  71.  If  A  =  Ai  U  A2  then  7t?„(p)(A)  = 

7p„(p)(Ai)  +  7-p„(p)(A2). 

Proof:  From  the  definition  of  7p.„(p)  (A)  we  have 

777, (P) (A)  =  ^2  TIp(-P) 

Pe  A 
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Applying  A  =  A-|  U  A2  and  associativity  of  +  allows  us  to 
conclude 

7p„(p)(A)  =  ^2  >(^1)+  ^2  >(^2) 

PiSAi  P1GA2 

Again  applying  the  definition  of  7p„(p)(-  ■  .),  we  have 

7p„(p)(A)  =  7p„(p)(Ai)  +  7p„(P)(-P2) 

■ 

Lemma  72.  If  D\  C  D[  and  D2  C  D2  then  Di  +  D2  C 
£'1  +  D'. 

Proof:  According  to  the  definition  of  addition  for  sets, 
we  have 

Di  +  D2  =  {<5i  +  62  |  S\  £  Di  A  52  £  D2} 

Consider  some  S  £  D 1  +  D2.  We  have  5  =  Si  +  S2  with 
Si  £  Di  and  S2  £  D2.  Since  D\  C  D\ ,  we  have  Si  £  D[. 
Similarly,  since  D2  C  D2,  we  have  S2  £  I)'2.  Since 

D'i  +  D'2  =  {tfi  +  S'2  |  <5;  £  D\  A  S'2£  D'2} 

we  have  6  =  Si  +  S2  £  D[  +  D2.  ■ 

B.  Bounding  Operation 

Lemma  26  (Soundness  of  Bounding  Operation). 

lVn(P)(A)  C  7Pn  (P)  (  L  A  J  n)  • 

Proof:  According  to  Definition  25,  there  are  two  cases 
for  [AJ  n.  If  | A  |  <  n  then  we  have  [AJ^  =  A  and  thus 

7p„(p)(^)  =  7p„(p)(L^J™)- 

If  | A |  >  n,  we  reason  by  induction  on  |A|.  Since  n  >  1, 
we  have  that  |A|  >2  and  thus  we  can  partition  A  into 
AiU{Pi,  P2}.  Applying  Definition  25  we  then  have  [AJ  „  = 
LA  1  U  {Pi  +  P2}J  „.  The  inductively-passed  set  has  size  one 
less  than  the  original,  allowing  us  to  apply  the  inductive 
hypothesis  to  conclude  the  following. 

Tp„(p)(Ai  U  {Pi  +  P2})  C  7p„(p)(LAi  U  {Pi  +  P2}Jn) 

Our  conclusion  will  follow  provided  we  can  show 

7p„(p)(A)  C  7p„(p)(Ai  U  {Pi  +  P2}) 

Lemma  71  allows  us  to  rewrite  this  to 

7t,„(p)(A)  Q  7'P„(p)(A1)  +  7'P„(p)({P1  +  P2])  (74) 

We  have  A  =  AiU{Pi,P2}  and  thus  by  Lemma  71  we 
have 

7p„(p)(A)  =  7p„(p)(Ai)  +  7p„(p)({-Pi,  P2}) 

By  Lemma  72,  we  will  have  (74)  provided  we  can  show 

7p„(p)(Ai)  C  7p„(p)(Ai) 

which  is  immediate,  and 

lvn{v){{Pu  P2})  Q  7p„(p)({Pi  +  P2}) 


The  latter  is  proven  by  applying  the  definitions  of 

7p„(p)({-Pi,  P2})  and  7p„(P)({-pi  +  P2}),  resulting  in  a 
goal  of 

Tp(-Pi)  +  lv{Pi)  Q  TIP  (-Pi  +  P2) 

which  follows  directly  from  Lemma  12.  ■ 

C.  Distributive  Operations 

The  soundness  proofs  for  the  majority  of  the  operations  on 
elements  of  Vn  (P)  are  sound  for  exactly  the  same  reason: 
the  operations  distribute  over  +,  allowing  us  to  reduce 
soundness  for  the  powerset  case  to  soundness  for  the  case  of 
a  single  probabilistic  polyhedron.  We  start  with  the  Lemma 
that  is  used  to  structure  such  a  proof. 

Lemma  73.  Consider  f  :  P  — »  P,  F  :  Vn  (P)  — >  Vn  (P), 
and  /b  :  Dist  — >  Dist.  Suppose  the  following  all  hold  for  all 

Si,  Pi. 

1)  f\Si  +  ...  +  Sn)  =  f(Si)  +  ...  +  f(Sn) 

2)  P({P1,...,P„})  =  {/(P1),...,/(P„)} 

3)  6  £  7p(P)  =7  /b(c>)  £  7p(/(P)) 

Then  S  £  7p„(p)(A)  implies  /b(d)  £  Tp„(p)(P(A)). 

Proof:  Suppose  S  £  7-p„(p)(A)  and  A  =  {Pi, . . . ,  Pn } . 
We  have  the  following  by  definition  of  7p„(p)(A). 

Tp„(p)(A)  =  7p(Pi)  +  •  ■  ■  +  7p  (Pn) 

Applying  the  definition  of  addition  on  sets,  we  obtain 

Tp(-Pi)  +  •  •  •  +  7p (Pn)  =  {<5i  +  ■  •  ■  +  Sn  |  Si  £  7p(Pi)} 

Thus,  we  have  that  S  =  (5i  +  . . .  +  Sn  where  <5,  £  7 p(Pj). 
By  premise  3  we  then  have  /b((5j)  £  7 r(f{Pi))  for  all  i. 

We  now  consider  Tp„(p)(P(A)).  By  premise  2  we  have 
that  this  is  7t’„(p)({/(Pi)>  •  •  • ,  f(Pn)})-  Applying  the  defi¬ 
nition  of  7-p„(P),  this  is  equal  to  >(/(Pi))-h  •  ■+lp(f(Pn))- 
Expanding  the  definition  of  +  for  sets,  we  have  that 

7-p„(p)(-P(A))  =  {<5i  +  •  •  •  +  Sn  |  Si  £  7p(/(Pj))} 

Since  fb(St)  £  7 p(/(Pi))  for  all  i  we  have  J2i(f(si))  e 
7Pr,  (p)  (P(A))  and  thus,  by  premise  1  we  have  /b(][7  Sf)  £ 
7p„(p)(7?(A))  and  thus  fb(S)  £  Tp„(p)(P(A))  as  desired. 

■ 

Lemma  74  (Soundness  of  Forget).  If  5  £  Tpn(p)(A)  then 
e  7-p„(P)(fi/(A)). 

Proof:  We  will  apply  Lemma  73  with  /b  =  A<5.  <5  f 
( fv{S )  —  {?/}),  /  =  A P.  f y(P),  and  P  =  AA.  fy(A).  Lemma 
7  gives  us  premise  3.  The  definition  of  f?y(A)  satisfies 
premise  2.  Let  V  =  fv(5)  —  {y}.  It  remains  to  show  premise 
1,  which  states 

(<5i  +  . . .  +  Sn)  \  V  =  I"  V  +  . . .  +  Sn  \  V 
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We  show  this  for  the  binary  case,  from  which  the  ?i-ary 
version  above  follows. 

(<$i  +  <52)  r  v  =  $1  r  v  +  62  r  v 

Expanding  the  definition  of  projection,  we  then  obtain  the 
following  goal. 

A av  £  Statey.  ^  (<5i  +  62)  (o')  = 

a’\(p'  fV=<7v) 

Aery  G  Statey.  E  <5i(ct') 

<r'|(cr'  IV— try ) 

+  A(Jy  G  Statey.  E  S2(a') 
<r'|(<r' \V=av) 

We  can  now  apply  the  definition  of  +  for  distributions  to 
the  right-hand  side  to  obtain  a  goal  of 

Aery  G  Statey.  ^  (<5i  +  S2)(a')  = 

<r'|(<r'  fy=<rv) 

Aery  G  statey.  f  E  6\(a')+  E  S2(a')  j 
cr'l  (o-'\V—aV)  (T/|((T/  \V=(T\r) 

These  functions  are  equal  if  they  give  equal  results  for  all 
inputs.  Thus,  we  must  show  the  following  for  all  ay. 

E  (*i+W)  = 

<j'\(<j'\V=<TV) 

(  E  Si o')  +  E  ooo) 

o' | (o'  \ V —o v )  o'\(o'\V=oy) 

Finally,  applying  the  definition  of  +  for  distributions  to  the 
left-hand  side  of  the  equality  yields 

E  (MO"')  +<J2(07))  = 

<j’\(ct'\V=<tv) 

(  E  ooo +  E  oo')) 

o'\(o'\V=Ov)  o'\(o'  \V =Oy) 

This  follows  by  associativity  and  commutativity  of  +.  ■ 

Lemma  75  (Soundness  of  Projection).  If  6  £  Tp„(p)(A) 
and  V  C/v(<5)  then  6  f  V  G  7p„(p)(A  j  V). 

Proof:  Inductive  application  of  Lemma  74  (Soundness 
of  Forget)  as  was  the  case  in  the  base  domain.  ■ 

Lemma  76  (Soundness  of  Assignment).  If  5  £  Tp„(p)(A) 
then  5  [. x  — >  E\  £  7pn(p)(A  [x  — >  E\). 

Proof:  As  in  Lemma  74,  we  apply  Lemma  73.  We  have 
premises  3  (by  Lemma  9)  and  2  (by  definition)  and  must 
show  premise  1.  This  means  showing  that 

(c>i  £-  S2)  [x  — >  E]  =  Si  [x  — >  E]  +  S2  \x  — >  E\ 

Expanding  the  definition  of  assignment,  we  must  show  that 
the  following 

A  ct.  E  (6i  +  $2)(t) 

r  |  t[x^{E\t}=(t 


is  equal  to 

(  Act.  E  (5i(t)  j  +  l  Act.  E  MO] 

V  r  |  t[x—*\E\t\=o  /  \  r  |  t[x—*IE^t]—o  / 

Again  applying  the  definition  of  +  for  distributions  and 
using  extensional  equality  for  functions  yields  the  following 
goal,  which  follows  by  associativity  and  commutativity  of 


Vcr.  (  E  (<5i(t)  +  <52(t)) 

\r  I  t\x — >^E^t]=o 


I  t[x^IE}t]  = 

E  00)  +  E  oo) 

r  |  r[x — ►  [Sjrj^cr  r  |  t[x—>^EJt]—o 


Lemma  77  (Soundness  of  Scalar  Product).  If  5  £  7-p„(p)(A) 
then  p  ■  5  £  7p„(p)(p  •  A). 

Proof:  This  proof  follows  the  same  format  as  the  others 
in  this  section.  We  apply  Lemma  9  with  the  definition  of 
scalar  product  for  powersets  and  Lemma  17.  We  must  show 

p  -  (Si+S2)  =  p-  Si+p-  S2 

Expanding  according  to  the  definition  of  scalar  product  and 
+  for  distributions,  we  obtain  the  following  as  a  goal. 

Act.  p  ■  (61(a)  +  62(a))  =  Act.  p  ■  61(a)  +  p  ■  S2(a) 

The  result  follows  by  distributivity  of  ■  over  +.  ■ 

Lemma  78  (Soundness  of  Conditioning).  If  6  £  7-p„(p)(A) 
then  6\B  £  7p„(p)(A  |  B). 

Proof:  Again  we  apply  Lemma  9,  this  time  using 
Lemma  15  to  satisfy  premise  3.  We  let  /b  =  A<5.  5\B, 
f  =  A P.  P  |  B,  and  F  =  AA.  A  |  B.  We  must  show 

(61  +  62)\B  =  61\B  +  62\B 

Applying  the  definition  of  conditioning  and  addition  for 
distributions,  we  have  to  show  the  following  for  all  a. 

if  [ B\a  then  (<5i  +  S2)(a)  else  0  = 

(  if  \B\a  then  5i(a)  else  0)  + 

(  if  \B\a  then  S2(a)  else  0) 

We  proceed  via  case  analysis.  If  \B\a  =  false  then  we 
have  0  =  0  +  0,  which  is  a  tautology.  If  \B\a  =  true  ,  we 
have  to  show 


Oi  +  S2)(a)  =  <5i(ct)  +  62(a) 

which  follows  directly  from  the  definition  of  +  on  distribu¬ 
tions.  ■ 
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D.  Other  Powerset  Lemmas 

We  now  show  the  lemmas  for  operations  in  the  powerset 
domain  that  do  not  immediately  follow  from  distributivity 
over  plus  of  the  operations  in  the  base  domain. 

Lemma  79  (Soundness  of  Product).  If  5  £  7p„(p)(A)  and 
S'  £  7v„(p)(P')  and  fv(A)  D  fv(P')  =  0  then  5  x  S'  £ 
7-p„(p)(A  x  P')- 

Proof:  Let  A  =  { l\ .. . . .  Pn } .  We  first  expand  defini¬ 
tions  in  our  goal,  obtaining 

S  x  S'  £  7p(Pi  x  P' )  +  . . .  +  7p {Pn  x  P') 

Applying  the  definition  of  addition  for  sets,  we  obtain  a  goal 
of 

5xfs{^^|)1e>(P!xP')} 

i 

This  holds  provided  we  can  find  Si  £  tp (Pi  x  P’)  such  that 
8x8'  =  Jfi  Si.  We  have  from  S  £  7p„(p)(A)  that  S  =  £A  Sj 
for  some  Sj  £  jp(Pj).  We  then  have  from  Lemma  13  and 
S'  £  7vn(p)(P')  and  fv(A)  n  jv(P’)  =  0  that  Sj  x  S'  £ 
7p (Pj  x  P')  for  all  j.  We  now  show  that  the  St  we  were 
searching  for  are  these  Sj  x  S'.  To  do  so,  we  must  show  that 
5x5'  =  fPjifij  x  S').  We  have  8  =  Sj  and  thus  the 
result  follows  by  distributivity  of  x  over  +,  which  we  show 
now. 

Goal:  x  distributes  over  +:  We  want  to  show  the  fol¬ 
lowing  when  domain (8 1)  =  domainifSf)  and  domain(S[ )  (7 
domain(S')  =  0. 

(<5i  +  c>2)  x  S'  =  (5i  x  S'  +  82  x  S' 

Expanding  the  definition  of  +  and  of  x,  we  obtain 

A(<7,  o').  (<5i(cr)  +  82 (cr))  ■  S' (o')  = 

A(cr,  o').  (<5i(cr)  •  S' (o')  +  62(0)  ■  S' (o')) 

This  holds  due  to  distributivity  of  ■  over  +.  ■ 

Lemma  80  (Soundness  of  Addition).  If  Si  £  7p„(p)(Ai) 
and  82  £  7'P„(p)(A2)  then  <)i  +  <S2  £  7'P„(p)(Ai  +  A2). 

Proof:  First  let  us  take  care  of  the  special  cases  that 
occur  when  iszero( Ai)  or  iszero( A2).  Without  the  loss 
of  generality  let  us  say  iszero( A2).  The  sum  is  defined 
to  be  identical  to  Ai.  Since  is2ero(A2),  it  must  be  that 
7-pri(P)(A2)  contains  only  the  zero  distribution  Onist.  there¬ 
fore  82  =  Onist-  Therefore  S-\  +  82  =  A  and  by  assumption, 
A  G  7p„(p)(Ai)  =  7t’„(p)(Ai  +  A2). 

In  the  case  where  Ai  and  A2  are  both  non-zero,  we  have 
Ax  +  A2  =  |Ai  U  A2J n.  Suppose  A  £  7-pn(P)(Ax)  and 
82  £  7p„(p)(A2).  By  Lemma  71  we  have  7p„(p)(Ai)  + 
Tvn{P) (A2)  =  Tp„(p)(Ai  U  A2).  The  set  7p„(p)(Ai)  + 
7t,„(p)(A2)  is  {^1  +  S'2  |  8)  £  7pn(P)(A1)  A  S'2  £ 

77;5ti(p)(A2)}.  Our  distributions  <5i  and  82  satisfy  these 
conditions  and  thus  are  in  7p„(p)(Ax  U  A2).  It  remains  to 
show  that  7pii(P)(AxUA2)  C  7-p„(p)(LAi U A2J„),  but  this 
is  exactly  Lemma  26.  ■ 


E.  Main  Soundness  Theorem  for  Powerset  Domain 

The  main  soundness  theorem  is  an  identical  restatement 
of  the  main  soundness  theorem  in  the  base  domain  and 
the  proof  is  likewise  identical,  save  for  replacement  of 
the  relevant  base  domain  definitions  and  lemmas  with  the 
powerset  ones.  The  only  corresponding  lemma  which  has 
not  yet  been  proven  follows  below. 

Lemma  81.  Let  A;  be  consistent  probabilistic  polyhedron 
sets,  that  is,  7p„(p)(A,)  ^  0.  Then,  Ai  +  A2  Ai  iff 
iszero(  A2). 

Proof:  The  proof  is  identical  to  the  Lemma  64,  re¬ 
placing  the  base  domain  lemmas  and  definitions  with  the 
powerset  ones.  ■ 

Theorem  24  (Soundness  of  Abstraction).  For  all  S,  S,  A,  if 
S  £  7pn(p)(A)  and  (( S ))  A  terminates,  then  [5]<5  terminates 
and  [S]$  £  7p„(p )(((S)}  A). 

Proof:  The  proof  is  identical  to  the  main  soundness 
proof  for  the  base  domain  (Theorem  6),  replacing  definitions 
and  lemmas  about  the  base  domain  abstraction  with  the 
corresponding  definitions  and  lemmas  about  the  powerset 
domain.  ■ 

Lemma  82  (Soundness  of  Normalization).  If  5  £  7-pn(p)(A) 
then  normal (5)  £  7-pn(p\(normal(A)). 

Proof:  Whenever  ||<5||  =  0,  the  normalization  in  the 
concrete  sense  is  undefined,  likewise  it  is  undefined  in  the 
abstract  sense.  So  let  us  assume  ||(S||  >  0. 

Let  to  =  J2i  m™ln  and  to  =  JA  m™ax.  By  assumption 
we  have  S  =  JA  Si  with 

Si  £  7p (P,)  (75) 

Thus  we  have  ||<5||  =  JA  ||<Jj||  and  we  conclude  to  = 
J2i  m“m  A  ||<5||  <  m™ax  =  to  via  (75). 

to  <  ||(5||  <  to  (76) 

Let  S'  =  normal (5)  =  pjyJ  =  t  due  to 

linearity  of  scalar  product.  Let  us  thus  show  that  £ 

7p(normal(P,)(TO,  to))  =  7p(normal(A))  which  would  con¬ 
clude  the  proof.  Let  us  write  If  =  normal (Pi)(rn,m)  and 
S i'  =  pjj&i.  We  must  thus  show  the  following. 

support  (Si’)  C  7c  (CV)  (77) 

s)?in  <  \support(Si’)\  <  s”?ax  (78) 

m™in  <  ||  (5c  ||  A  m™ax  (79) 

Vtr  £  support(Si')  .  p"lm  <  8i'(o)  <  p“ax  (80) 

Claim  (77)  holds  trivially  as  support(Si>)  =  support(Si), 
Ci 1  =  Ci,  and  (75).  Claim  (78)  holds  due  to  the  same 
reasoning. 


37 


For  (79),  in  the  case  where  m  >  0,  we  reason,  via  (76), 
as  follows. 


m™m  <  ||&||  < 


^  min  i  mma: 

-in- s  w  »i-» s  i r 


nr 

m 


< 


M*1 


< 


m 


^min  ™max 

mf=  -^r-  <  Ill'll  < 

m  m 


If  m  =  0,  the  definition  of  normalization  makes  m“ax 
1 .  which  is  also  sound  as  all  distributions  have  mass  no  more 
than  1. 

The  (80)  claim  is  shown  using  reasoning  identical  to  the 
mass  claim  above.  ■ 


Lemma  28  (Soundness  of  Simple  Maximal  Bound  Es¬ 
timate).  If  5  £  lv„(v){{Pi})  and  P  =  ff,  Pi  then 
max„  5(a)  <  pmax. 

Proof:  By  assumption  we  have  6  =  JT  <5,  with  Si  £ 
7p (Pf)  thus  by  Lemma  12  (Soundness  of  Plus),  we  have 
<5  G  7p (J2iPi)  =  7p(-P).  thus  for  every  er  £  support(S), 
5(a)  <  pmax,  hence  max,  5(a)  <  pmax.  ■ 

The  above  lemma  shows  soundness  of  the  very  simple 
method  of  estimating  the  maximum  probability  but  in  the 
implementation  we  use  the  method  based  on  poly  partition¬ 
ing  and  the  following  lemma. 

Lemma  32.  maxpp( A)  =f  ma,xa£R  Amax  (a)  = 

max^  Amax  (a)  where  C  is  a  poly  partition  of  A  and  R 
is  a  representative  set  of  C. 

Proof:  Let  C  be  the  poly  partition  of  A  =  {Cj}  as  in 
the  statement  of  the  lemma.  Let  us  first  show  a  claim:  if 
cr,  er'  €  L  £  C  then 

A  =  {C  £  A  |  a  e  7c (C)}  =  {C  £  A  \  o' £  7c(C)}  =  B 

(81) 

Let  C  £  A.  Thus  a  £  jc(C)  so  by  Definition  31  (2), 
we  have  er  £  jc(L')  for  some  L'  £  C.  By  (1)  it  must 
be  that  L  =  L'  and  by  (3),  we  have  7c (L)  =  jc(L')  C 
7c(C').  Therefore  er'  £  7c(C)  and  thus  C  £  B,  showing 
A  C  B.  The  other  direction  is  identical,  concluding  A  =  B 
as  claimed.  □. 

Now  we  can  get  back  to  the  main  lemma.  Let  a*  be  the 
state  with  Amax  (cr*)  =  maxo- A max  (cr) .  Thus  er*  £  jc(L) 
for  some  L  £  C,  by  Definition  31  (2).  Let  a ^  be  any 
representative  of  L,  that  is  ctl  £  Jc(L)- 


Amax  (cr*)  =  ^  Tf ax  (a*) 

=  *  E  pr 

i  I  <r*67c(Ci) 

=  E  pf 

i  |  o-l  67€  (Ci) 

=  EpraxK) 

i 

=  Amax  (aL) 


[  by  (81)  ] 


Now  we  see  that  maxff  Amax  (er)  =  Amax(cr*)  = 
Amax  (0 l)  =  maxPp  (A)  as  claimed.  ■ 

Before  we  prove  the  security  theorem,  let  us  show  that 
the  definition  of  abstract  conditioning  on  a  state  is  sound. 


Lemma  83.  If  5  £  jp(A)  and  ay  £  Statey  with  V  C.f\’(5) 
then  5\ov  £  7p„( p)(A  |  ay) 

Proof:  Recall  the  definition  of  A  |  ay. 


A\av=A\B 

With  B  =  f\xeV  (x  =  oy(x)).  Let  us  show  that  <5|cry  = 
5 1 B,  the  rest  will  follow  from  Lemma  78. 

The  definition  of  5\oy  is  as  follows. 

5\a  =  A  a.  if  a  \  V  =  ay  then  5(a)  else  0 

Meanwhile,  <5|B  is  defined  as  follows. 

6\B  =  A  a.  if  \B\a  =  true  then  5(a)  else  0 

The  correspondence  is  immediate  as  \B\a  =  true  if 
and  only  if  a  \  V  =  ay  as  per  construction  of  B.  ■ 

Theorem  35  (Soundness  for  Threshold  Security).  Let  5 
be  an  attacker’s  initial  belief.  If  5  £  7p„( p)(A)  and 
tsecuret(S,  A),  then  S  is  threshold  secure  for  threshold  t 
when  evaluated  with  initial  belief  5. 

Proof:  Let  us  consider  the  contrapositive.  That  is, 
assuming  5  £  7p„(p)(A),  if  S  is  not  threshold  secure  for  t 
and  initial  belief  5,  then  it  is  not  the  case  that  tsecuret(S,  A). 

Let  ^2  =  [S']  <5  and  5 3  =  S2  |  L.  Since  S  is  not 
secure,  we  have  ol  £  support(5f)  and  a'H  £  State# 
with  (normal((52|cri)  [  H))(o'h)  >  t.  This  implies  that 
(52\ol)  r  H  jn  0Dist  and  therefore  <52|crL  ^  0Dist  as 
projection  preserves  mass. 

If  [S]A  is  not  terminating,  then  we  are  done  as  termi¬ 
nation  is  a  condition  for  tsecuret(S,  A).  So  let  us  assume 
((S))  A  is  terminating.  Let  A2  =  ((S))A.  By  Theorem 
24,  we  have  52  £  7p„(p)  (A2).  By  Lemma  83,  62 \ a /-,  £ 
7#ii(P)(A2  I  a if).  Therefore  not  iszero( A2  |  aL).  Continu¬ 
ing,  by  Lemma  75,  (82\ol)  \  H  £  7p„(p)((A2  |  aL)  \  H) 
and  finally,  by  Lemma  82,  we  have  5 4  =  normal((<)2|erL)  [ 
H)  £  7p„(p)(normal((A2  |  a l)  \  H)).  Let  A4  = 
normal((A2  |  aL)  \  H). 

By  Remark  30,  we  have  54(a'H)  <  maxCT  A”ax  (a)  and 
by  Lemma  32  we  have  max^  A™ax  (a)  =  maxpp  (A4).  But 


38 


6a(cf'h)  >  t  so  maxpp  (A4)  >  t,  a  potential  failure  of 
tsecuret(S ,  A). 

To  finish  the  proof  we  need  to  make  sure  that  07  was 
indeed  a  valid  witness  to  the  failure  of  tsecuret(S ,  P\) .  Let 
A3  =  {P"}  =  A2  \  L.  By  Lemma  75,  we  have  £3  € 
7’PTl(P)(A3)  so  S3  =  (5-  with  5-  e  7p (P")-  since  G 


support^Sz)  it  must  be  that  £3(07,)  >  0  and  thus  £>'(07,)  >  0 
for  at  least  one  i.  Thus  07,  G  support (<5-)  C  jc(C-')  for  at 
least  one  i  and  therefore  07,  G  7p(C)({Cf})-  Also,  we  have 
already  shown  that  not  iszero( Ao  |  07),  thus  07  is  indeed 
the  witness  as  needed.  ■ 
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